Prohibition of DUnit GUI Runner for Delphi XE2 in 1.2.4 - intention or accident ?

Issue #338 closed
Arioch The created an issue

Using Delphi XE2 and running Build.exe on the Tag 1.2.4 branch i was surprised the building ending silently without tests running.

I did not set “console tests” check box so i expected the familiar DUnit window to pop up and wait for my attention. Did not happen.

The reason seems to be c521c01ed8b6b466d59c3c00701b9407930a2aa6 of 12.06.2018 which introduced two {$IFDEF DELPHI2010} clauses into Spring.TestRunner.pas instead of {$IFDEF DELPHI2010_up}.

Neither the commit nor the sources have any comments they were circumventing post-2010 bugs.

So it does look as accident, but i am not sure.

The Win32 Debug build of basic Spring.Test.exe (from under GUI but without debugger attached) seems to finish “all green”.

I re-built 1.2.4 clean, and i see Win32 Release (but not debug) build failing tests! Damned details:

  1. Build.exe fails to detect it and give any warning!
  2. The error information gets spitted into console (stdout) and then closed in a blink. It is saved nowhere! Okay, there is huge XML report but will a human being ever read it? I think the non-GUI runner should save it’s console output when run like that, with non-persistent console window. Maybe it also should set some exit error code that Build.exe will know of failures.

Comments (39)

  1. Arioch The reporter

    Win 32 Release build console output.

    Time: 0:00:00.583
    
    FAILURES!!!
    Test Results:
    Run:          1432
    Failures:        0
    Errors:          8
    There were 8 errors:
      1) SimpleUniqueKeys: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      2) OrderingIsStable: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      3) NilComparerIsDefault: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      4) CustomComparer: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      5) SimpleUniqueKeys: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      6) OrderingIsStable: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      7) NilComparerIsDefault: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
      8) CustomComparer: EArgumentOutOfRangeException
         at
          "Specified argument was out of the range of valid values: count"
    

  2. Arioch The reporter

    Even if i select “pause after each step” - there is NO pause after test runs.

    Perhaps Build.exe is confused by thew test not requiring user interaction, but maybe it could at least check exit code ?


    Win64 Debug build of tests also spawned .RSM file, but Win32 Debug build did not. Dunno if this inconsistency is intended.


    Build.exe did not indicate completing installation and testing otherwise than hiding an hourglass (IF the mouse happens to be over its window).

    Some more explicit visual clue, like OK `ShowMessage` or at least re-Enabling disabled buttons would havd been handy


    Win64 both Debug and Release core tests seem to finish ok

  3. Arioch The reporter

    re-enabled DUnit GUI runner, and compiled it for Win32 Release, it reported visibly the same 8 errors.

    So, probably an accident.

    All those errors seems to have the same “location”, for example

     EArgumentOutOfRangeException
    at  $0095294B
    Specified argument was out of the range of valid values: count
    

  4. Arioch The reporter

    D:\DelphiProjects\Libs\Spring4D\Tests\Source\Base\Spring.Tests.Collections.Extensions.pas
    procedure TTestOrderBy.SimpleUniqueKeys;

    it fails within destructors chain.

    Tried both ordering - the “2” mark is always reached and “3” is reached never.

      showmessage('1');
      source := nil;
      showmessage('2');
      query := nil;
      showmessage('3');
    
      -----
    
      showmessage('1');
      query := nil;
      showmessage('2');
      source := nil;
      showmessage('3');
    

  5. Arioch The reporter

    After all reshufflings the error is now “at” $0095295B

    00952935 E846FDBBFF       call Guard.RaiseArgumentOutOfRangeException
    0095293A 85F6             test esi,esi
    0095293C 7C09             jl $00952947
    0095293E 8B4340           mov eax,[ebx+$40]
    00952941 2BC7             sub eax,edi
    00952943 3BF0             cmp esi,eax
    00952945 7E04             jle $0095294b
    00952947 33C0             xor eax,eax
    00952949 EB02             jmp $0095294d
    0095294B B001             mov al,$01
    0095294D 84C0             test al,al
    0095294F 750A             jnz $0095295b
    00952951 B8582A9500       mov eax,$00952a58
    00952956 E825FDBBFF       call Guard.RaiseArgumentOutOfRangeException
    >>>>>>>>>>>>> 0095295B 85F6             test esi,esi      <<<<<<<<<
    0095295D 0F84A3000000     jz $00952a06
    00952963 56               push esi
    00952964 8D45FC           lea eax,[ebp-$04]
    00952967 B901000000       mov ecx,$00000001
    0095296C 8B1500DF8F00     mov edx,[$008fdf00]
    00952972 E87571ABFF       call @DynArraySetLength
    00952977 83C404           add esp,$04
    0095297A 6A00             push $00
    0095297C 56               push esi
    0095297D 8D55FC           lea edx,[ebp-$04]
    00952980 8D433C           lea eax,[ebx+$3c]
    00952983 8BCF             mov ecx,edi
    00952985 E89A5F0100       call {Spring.Collections}TArrayManager<System.Generics.Collections.TPair<System.Integer,System.Integer>>.Move
    0095298A 8D4344           lea eax,[ebx+$44]
    0095298D FF00             inc dword ptr [eax]
    

    :7522c5af KERNELBASE.RaiseException + 0x58
    Spring.Tests.Collections.Extensions.{Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal(???,???,True)
    Spring.Tests.Collections.Extensions.{Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal(???,???,True)
    :0095295b {Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal + $63
    Spring.Tests.Collections.Extensions.{Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal(???,???,True)
    Spring.Tests.Collections.Extensions.{Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal(???,???,True)
    :0095295b {Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.DeleteRangeInternal + $63
    :00952e67 {Spring.Collections.Lists}TList<System.Generics.Collections.TPair<System.Integer,System.Integer>>.Clear + $17
    :00561d70 TEnumerableBase._Release + $C
    :0040b2df @IntfClear + $13
    :00561d70 TEnumerableBase._Release + $C
    :0040b2df @IntfClear + $13
    :00561d70 TEnumerableBase._Release + $C
    :0040b2df @IntfClear + $13
    :004c147d TTestCase.Invoke + $D
    

  6. Arioch The reporter

    Here be dragons 😞 XE2 starts ICE-ing heavily anywhere aroung S4D generic collections and ARC

    D:\DelphiProjects\Libs\Spring4D\Source\Base\Collections\Spring.Collections.Lists.pas

    procedure TList<T>.DeleteRangeInternal(index, count: Integer; doClear: Boolean);
    var
      oldItems: TArray<T>;
      tailCount,
      i: Integer;
      defaultItem: T;
      _s: string;
    begin
    {$IFDEF SPRING_ENABLE_GUARD}
      if Self = nil
         then _s := '<<NIL>>'
         else _s := #13#10 + QualifiedClassName + ' @ $' + IntToHex(Cardinal(Pointer(Self)),8) + ' refCnt=' + IntToStr(RefCount);
      _s := Format('; %s'#13#10#9#9' with index = %u = $%x; count = %u = $%x',
        [_s, index, index, count, count]);
      Guard.CheckRange((index >= 0) and (index < fCount), 'index' + _s );
      Guard.CheckRange((count >= 0) and (count <= fCount - index), 'count' + _s );
    {$ENDIF}
    

    After some instrumenting added

    SimpleUniqueKeys: EArgumentOutOfRangeException
    at  $0095693B
    Specified argument was out of the range of valid values: count; 
    Spring.Collections.Lists.TList<System.Generics.Collections.TPair<System.Integer,System.Integer>> @ $02D82F00 refCnt=0
             with index = 0 = $0; count = 1634932 = $18F274
    
    OrderingIsStable: EArgumentOutOfRangeException
    at  $0095693B
    Specified argument was out of the range of valid values: count; 
    Spring.Collections.Lists.TList<System.Generics.Collections.TPair<System.Integer,System.Integer>> @ $02D82FB0 refCnt=0
             with index = 0 = $0; count = 1635024 = $18F2D0
    
    NilComparerIsDefault: EArgumentOutOfRangeException
    at  $0095693B
    Specified argument was out of the range of valid values: count; 
    Spring.Collections.Lists.TList<System.Generics.Collections.TPair<System.Integer,System.Integer>> @ $02D834D8 refCnt=0
             with index = 0 = $0; count = 1635044 = $18F2E4
    

    …and so forth

  7. Arioch The reporter

    This fails too. So it is about generics-inside-generics and XE2

      RPlain<T> = record a: T; b: integer; end;
      RPlain = RPlain<integer>;
    
        i := TCollections.CreateList<RPlain>([ d1 ]);
    

  8. Arioch The reporter

    :7522c5af KERNELBASE.RaiseException + 0x58
    Project46.{Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal(???,???,True)
    Project46.{Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal(???,???,True)
    :004bfc17 {Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal + $63
    Project46.{Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal(???,???,True)
    Project46.{Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal(???,???,True)
    :004bfc17 {Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.DeleteRangeInternal + $63
    :004c0123 {Spring.Collections.Lists}TList<Project46.RPlain<System.Integer>>.Clear + $17
    :004b3e34 TEnumerableBase._Release + $C
    :0040ae77 @IntfClear + $13
    :004e177d Project46 + $41
    :7667343d kernel32.BaseThreadInitThunk + 0x12
    :77ad9812 ntdll.RtlInitializeExceptionChain + 0x63
    :77ad97e5 ntdll.RtlInitializeExceptionChain + 0x36
    

    program Project46;
    
    {$APPTYPE CONSOLE}
    
    {$R *.res}
    
    uses
      System.SysUtils,
      Spring.Collections;
    
    type
      RPlain<T> = record a: T; b: integer; end;
      RPlain = RPlain<integer>;
    
    var
       d1: RPlain;
    
    procedure Bang;
    var i: IUnknown;
    begin
      d1.a := 10; d1.b := -10;
      i := TCollections.CreateList<RPlain>([ d1 ]);
    
      i := nil; // desturctor is NOT called here yet
    end;
    
    begin
      try
    
        Bang;
    
      except
        on E: Exception do
          Writeln(E.ClassName, ': ', E.Message);
      end;
    
      Readln;
    end.
    

  9. Arioch The reporter

    It is some wicked compiler bug about proc params overriding. Like in “good old” days of Delphi 2.0

    {$O-} fixes it.

    procedure TList<T>.DeleteRangeInternal(index, count: Integer; doClear: Boolean);

    Entered with correct registers/params, but immediately after “begin” screws ECX/count


    The question remains though about that passing behind radar, given the extensive testing suits you develope, and given those tests being default part of Build.exe, so tests expectedly were run by both you and any other XE2 user of 1.2.4 (if any)

  10. Arioch The reporter

    Spring.Collections.Lists.pas.618: begin
    004C19F8 55               push ebp
    004C19F9 8BEC             mov ebp,esp
    

    Here we have 01=True on stack, EDX = 0 and ECX = 3 (count, valid)

    004C19FB 83C48C           add esp,-$74
    004C19FE 53               push ebx
    004C19FF 33DB             xor ebx,ebx
    004C1A01 895D8C           mov [ebp-$74],ebx
    004C1A04 895D90           mov [ebp-$70],ebx
    004C1A07 895DBC           mov [ebp-$44],ebx
    004C1A0A 895DCC           mov [ebp-$34],ebx
    004C1A0D 895DC8           mov [ebp-$38],ebx
    004C1A10 895DC4           mov [ebp-$3c],ebx
    004C1A13 895DC0           mov [ebp-$40],ebx
    004C1A16 895DF0           mov [ebp-$10],ebx
    004C1A19 895DDC           mov [ebp-$24],ebx
    

    Creating 9 zeroed local pointers? String, interfaces, visible and invisible…

    var
      oldItems: TArray<T>;
      tailCount,
      i: Integer;
      defaultItem: T;
      _s: string;
    
    // T = RPlain<T=integer> = record a: T=integer; b: integer; end;
    

    004C1A1C 894DF4           mov [ebp-$0c],ecx // == 3, count
    004C1A1F 8955F8           mov [ebp-$08],edx // == 0, index
    004C1A22 8945FC           mov [ebp-$04],eax // == $61DAC8, Self
    

  11. Arioch The reporter

    Further, my (and yours) instrumentation kicks in and the ECX gets overriden by Long String Assign proc (for example, or any other one)

    004C1A25 33C0             xor eax,eax
    004C1A27 55               push ebp
    004C1A28 68AE1C4C00       push $004c1cae
    004C1A2D 64FF30           push dword ptr fs:[eax]
    004C1A30 648920           mov fs:[eax],esp
    Spring.Collections.Lists.pas.620: if Self = nil
    004C1A33 837DFC00         cmp dword ptr [ebp-$04],$00
    004C1A37 750F             jnz $004c1a48
    Spring.Collections.Lists.pas.621: then _s := '<<NIL>>'
    004C1A39 8D45DC           lea eax,[ebp-$24]
    004C1A3C BAC81C4C00       mov edx,$004c1cc8
    004C1A41 E86265F4FF       call @UStrLAsg
    004C1A46 EB6A             jmp $004c1ab2
    

    index is [EBP-8] now, and that is how it gets referred too when Optimization is OFF

    Spring.Collections.Lists.pas.625: Guard.CheckRange((index >= 0) and (index < fCount), 'index' + _s );
    004C1B05 837DF800         cmp dword ptr [ebp-$08],$00
    004C1B09 7C0B             jl $004c1b16
    004C1B0B 8B45F8           mov eax,[ebp-$08]
    004C1B0E 8B55FC           mov edx,[ebp-$04]
    004C1B11 3B4240           cmp eax,[edx+$40]
    004C1B14 7C04             jl $004c1b1a
    004C1B16 33C0             xor eax,eax
    004C1B18 EB02             jmp $004c1b1c
    004C1B1A B001             mov al,$01
       ............
    

    and same for count which is [EBP-12]

    Spring.Collections.Lists.pas.626: Guard.CheckRange((count >= 0) and (count <= fCount - index), 'count' + _s );
    004C1B3D 837DF400         cmp dword ptr [ebp-$0c],$00
    004C1B41 7C0E             jl $004c1b51
    004C1B43 8B45FC           mov eax,[ebp-$04]
    004C1B46 8B4040           mov eax,[eax+$40]
    004C1B49 2B45F8           sub eax,[ebp-$08]
    004C1B4C 3B45F4           cmp eax,[ebp-$0c]
    004C1B4F 7D04             jnl $004c1b55
    004C1B51 33C0             xor eax,eax
    004C1B53 EB02             jmp $004c1b57
    004C1B55 B001             mov al,$01
    

  12. Stefan Glienke repo owner

    As for the unit tests - I never use the GUI runner but TestInsight - also I did not do any extensive testing for 1.2.4 because that was just 10.4 support, so this might have been broken for ages.

    Unfortunately I don’t care to fix or work around some 10 versions ago (I can repro in XE as well) compiler bugs unless they are trivial and this one looks like it requires quite some investigation - if you care, be my guest - otherwise: won’t fix - I am busy with getting 2.0 ready and probably that will wait without enough shit from those old compilers which I ignored until now. 😞

    Update: well at least in XE I can’t see these tests failing anymore with 2.0/develop and with $O+ - that’s a relief

  13. Arioch The reporter

    With Optimizations ON it becomes a bit different.

    Spring.Collections.Lists.pas.618: begin
    004C1314 55               push ebp
    004C1315 8BEC             mov ebp,esp
    004C1317 83C4A8           add esp,-$58
    004C131A 53               push ebx
    004C131B 56               push esi
    004C131C 57               push edi
    004C131D 33DB             xor ebx,ebx
    004C131F 895DA8           mov [ebp-$58],ebx
    004C1322 895DAC           mov [ebp-$54],ebx
    004C1325 895DD8           mov [ebp-$28],ebx
    004C1328 895DE8           mov [ebp-$18],ebx
    004C132B 895DE4           mov [ebp-$1c],ebx
    004C132E 895DE0           mov [ebp-$20],ebx
    004C1331 895DDC           mov [ebp-$24],ebx
    004C1334 895DFC           mov [ebp-$04],ebx
    004C1337 895DF8           mov [ebp-$08],ebx
    

    Same except for stack frame size here.

    004C133A 8BF1             mov esi,ecx
    004C133C 8BFA             mov edi,edx
    004C133E 8BD8             mov ebx,eax
    

    This instead of prior

    004C1A1C 894DF4 mov [ebp-$0c],ecx // == 3, count

    004C1A1F 8955F8 mov [ebp-$08],edx // == 0, index

    004C1A22 8945FC mov [ebp-$04],eax // == $61DAC8, Self

    004C1340 8D75F0           lea esi,[ebp-$10]
    

    OOOOOPS !!!

    Something pointer to some local on-stack var gets thrown into temporary count storage.

    But ESI register seems not used later!

    004C1343 33C0             xor eax,eax
    004C1345 55               push ebp
    004C1346 684E154C00       push $004c154e
    004C134B 64FF30           push dword ptr fs:[eax]
    004C134E 648920           mov fs:[eax],esp
    Spring.Collections.Lists.pas.620: if Self = nil
    004C1351 85DB             test ebx,ebx
    004C1353 750F             jnz $004c1364
    Spring.Collections.Lists.pas.621: then _s := '<<NIL>>'
    004C1355 8D45F8           lea eax,[ebp-$08]
    004C1358 BA6C154C00       mov edx,$004c156c
    004C135D E8466CF4FF       call @UStrLAsg
            ...........
    

    Spring.Collections.Lists.pas.623: _s := Format('.... count = %u = $%x', [ ... ]
    004C13C4 8D45D8           lea eax,[ebp-$28]
             .......
    004C13E0 8975C8           mov [ebp-$38],esi
    004C13E3 C645CC00         mov byte ptr [ebp-$34],$00
    004C13E7 8975D0           mov [ebp-$30],esi
    004C13EA C645D400         mov byte ptr [ebp-$2c],$00
    

  14. Arioch The reporter

    And here we run into guard (if guard is off - then we run into good old AV).

    Spring.Collections.Lists.pas.626: Guard.CheckRange((count >= 0) and (count <= fCount - index), 'count' + _s );
    004C1436 85F6             test esi,esi
    004C1438 7C09             jl $004c1443
    004C143A 8B4340           mov eax,[ebx+$40]
    004C143D 2BC7             sub eax,edi
    004C143F 3BF0             cmp esi,eax
    004C1441 7E04             jle $004c1447
    004C1443 33C0             xor eax,eax
    004C1445 EB02             jmp $004c1449
    004C1447 B001             mov al,$01
    004C1449 84C0             test al,al
    004C144B 7518             jnz $004c1465
    

  15. Arioch The reporter

    Fixing this specific bug will be as easy to turning off optimization for that specific functin oand that specific compiler.

    Just that is not true fix.

    The real question is what other functions are affected and can generate shroedinbugs, emerging and submerging with no clear pattern.

  16. Stefan Glienke repo owner

    What is the LAST version where it still happens ?

    Can’t tell - only have XE and >=10.1 on my local machine right now.

    Since this only has been found after such a long time is an indication to be that the severity is not critical althought its a nasty bug no question.

    By the way placing $O- anywhere within Spring.Collections.Lists had no effect (in XE) - only placing it in Spring.Tests.Collections.Extensions.pas helped because thats the place the defect list gets generated obviously - that might fix the unit tests but not the issue a user might have.

    Again if you care to investigate I would be pleased if you rather give develop a try under XE2 and provide any feedback - that would be more valuable than banging on a version that is becoming obsolete soon 🙂

  17. Arioch The reporter

    Yep, i have my intermediate result too. I have to place {$o-} immediately before the end. in my .DPR but not after.

    Guess the generic code gets actually instantiated at the unit end.

    This perhaps can help with this particular bug IF the method can be moved into some class in some other unit or something along the line. But such a fix would probably be no less fragile than the bug itself.

    On a somewhat brighter note this might open quick-fixes for XE2 generics bugs, granted i never saw so many of them than when trying s4d collections. Seems you really push the compiler to limits 🙂


    Fixing this problem actually is no brainer, just pack the params into record.

    I am more concerned about same param bombing in other methods.

    How to specifically to find OTHER instances, what specifically causes this one?

    Questions…

  18. Arioch The reporter

    that is becoming obsolete soon

    looking at the timetable for alpha 2 and alpha 1 i doubt it.

    And i truly doubt XE2 will manage it. real LOTS of ICEs anytime i tried real use of s4d colections made me finally avoid them. There is no point in using s4d complexities if in the end you only dare to use same features as RTL itself provides :-(

  19. Arioch The reporter

    But to the unit tests…

    0. so supression of GUI runner was an acicdent and i can get it back on, right?

    1. can you call on alleged users of S4D if they have Delphi versions XE3 to 10.1 so they can test it. Thin chance, but why not try?
    2. it still bugs me that tests were showing clear crash - but Build.exe failes to stop and inform the user. I think THAT is the problem, regardless of compiler bugs. And i think it is not 1.2.4-only too :-)

  20. Stefan Glienke repo owner

    2.0 is way more stable because I stripped a lot of the fat from the generics - also I said this before: spring4d alpha is just called alpha because I could not guarantee no further changes to the API itself at that point - it does not tell anything about its quality - it has been rock solid and being used in production by several parties.

    Anyhow time of us all is limited so I rather don’t spend it to fix an issue for a person that is not using the library anyway 😉

  21. Arioch The reporter

    Can you tell me what // workaround for RSP-20683 is about ?

    Emba prohibited public access to new bugtracker, and i am not fond of creating yet another online account that i will not make much use of.

    This line was a clear beacon and….

    Spring.Collections.Lists.pas.646: defaultItem := Default(T);
    004C0C71 33C0             xor eax,eax
    004C0C73 8906             mov [esi],eax
    004C0C75 894604           mov [esi+$04],eax
    Spring.Collections.Lists.pas.647: Changed(defaultItem, caReseted);
    004C0C78 B105             mov cl,$05
    004C0C7A 8BD6             mov edx,esi
    

    The first use of ESI as anything different than that count var.

  22. Stefan Glienke repo owner

    Codegen error when passing default(T) to a generic method, and T is a method pointer

  23. Arioch The reporter

    I get your point about XE2 support, though it would be nice if you put some testing matrix online so that “XE2 is NOT tested” right in your face.

    LOL, so like old good times when i was JediVCL Delphi 5 maintainer, the first and the last 😃 Guess i am a man from a backward side 🙂

    But on a serios note, the very infrastructure about installatin emanates the “industry best practices, rock solid and bulletproof” vibe. And that is why it would be a last place to notice failures. Would it be not to “WTF, where is GUI???” - and i probably would never notice this bug too.

    So i wish you focus on the building/testing infrastructure, it will benefit S4D 2.x and it seems a potential tarpit hidden in dark corner to me. It bugs me more than this compiler glitch itself. Bugs happen, everyday. But failing test suite is another magnitude proiblem.

  24. Arioch The reporter

    Codegen error when passing default(T)

    Is there a simple way to test it ? Maybe XE2 is not affected by it, for example.

    Update: it was affected.

  25. Arioch The reporter

    I mean, every time i change this Spring.Collections.Lists i have to clean-build my test project, because simple f9/make/compile results in

    [DCC Fatal Error] Spring.Collections.Lists.pas(1608): F2084 Internal Error: AV086221A0-R0000000C-0

    But, if i merely comment out defaultItem := Default(T); - then neither ICE seem to happen any more, nor count gets destroyed (the LEA ESI thing is removed)

    Workarounds… whan a shaky person starts falling from a bridge onto the left - give him a heavy kick to the right. Nice and dandy, but when another person was walking shakey on the bridge right side - it would exactly push him over… That was my gut feeling and it found something.

    So, i am starting to thing what that specific workaround was (comparing wit hclean code before it) and so if XE2 (XE3? XE4?..) can live without it…

  26. Stefan Glienke repo owner

    I am working on a CI system, don’t worry - please point me to other Delphi open source libraries supporting 12 Delphi versions and up to 10 target platforms that have extensive unit tests which are run continously - maybe I can take an example from them 😉

    Not so easy as with other languages/compilers that you can just run in the cloud on some service without a hefty installation.

    As for RSP-20683 - see https://stackoverflow.com/questions/50708095/ilistt-crashes-when-t-is-an-event-handler - probably it did not happen on older versions - but I am not putting IFDEFs for every fucking compiler issue into the code base. However it seems that not calling Default(T) prevents the bad codegen to happen - maybe it might even be a good idea to put defaultItem as static field into the class to avoid the stack space on every use.

    I mean, every time i change this Spring.Collections.Lists i have to clean-build my test project, because simple f9/make/compile results in

    [DCC Fatal Error] Spring.Collections.Lists.pas(1608): F2084 Internal Error: AV086221A0-R0000000C-0

    Welcome to my world. Don’t ask me how often my IDE or the compiler die every day when working on the library - but that is usually when you change generic code and almost never when you just use the collections.

  27. Arioch The reporter

    I find one half-ugly half-fix. DIVIDE ET EMPERRRRA ! 😃

    Half-fix is because XE2 still ICEs like hellstorm. 😞

    Half ugly is because it pollutes interface section of the unit. But hopefully you can move it to some utilities unit.

    unit Spring.Collections.Lists;
    .....
    
    // this can NOT be unit-local, or all kinds of errors, including very funny one, ensue
    type dummy = record  
       class procedure makeDefault<T>(out V: T); static;
    end;
    
    implementation
    
    ....
    
    procedure TList<T>.DeleteRange(index, count: Integer);
    begin
      DeleteRangeInternal(index, count, False);
    end;
    
    class procedure dummy.makeDefault<T>(out V: T);
    begin
      V := Default(T);
    end;
    
    procedure TList<T>.DeleteRangeInternal(const index, count: Integer; const doClear: Boolean);
    var
      oldItems: TArray<T>;
      tailCount,
      i: Integer;
      defaultItem: T;
    begin
    
    .....
    
      if doClear then
      begin
        // workaround for RSP-20683
    //    defaultItem := Default(T);
        dummy.makeDefault<T>(defaultItem);
        Changed(defaultItem, caReseted);
      end;
    

    Didn’t run tests yet (and i just can not test non-XE2), but at the first glance looks promising.

    Spring.Collections.Lists.pas.627: begin
    004BFE08 55               push ebp
    004BFE09 8BEC             mov ebp,esp
    004BFE0B 83C4F0           add esp,-$10     --- YAHOO!!!!! cut the slack!
    004BFE0E 53               push ebx
    004BFE0F 56               push esi
    004BFE10 57               push edi
    004BFE11 33DB             xor ebx,ebx
    004BFE13 895DFC           mov [ebp-$04],ebx
    004BFE16 8BF1             mov esi,ecx
    004BFE18 8BFA             mov edi,edx
    004BFE1A 8BD8             mov ebx,eax
    
    ....here once was the dreaded " lea esi,[ebp-$10] " bomb.
    
    004BFE1C 33C0             xor eax,eax
    004BFE1E 55               push ebp
    004BFE1F 6832FF4B00       push $004bff32
    004BFE24 64FF30           push dword ptr fs:[eax]
    004BFE27 648920           mov fs:[eax],esp
    Spring.Collections.Lists.pas.630: Guard.CheckRange((index >= 0) and (index < fCount), 'index' );
    004BFE2A 85FF             test edi,edi
    004BFE2C 7C05             jl $004bfe33
    
    .............
    
    Spring.Collections.Lists.pas.652: if doClear then
    004BFED5 807D0800         cmp byte ptr [ebp+$08],$00
    004BFED9 7414             jz $004bfeef
    Spring.Collections.Lists.pas.656: dummy.makeDefault<T>(defaultItem);
    004BFEDB 8D45F4           lea eax,[ebp-$0c]
    004BFEDE E8DD2F0000       call dummy.makeDefault<Project46.RPlain<System.Integer>>
    Spring.Collections.Lists.pas.657: Changed(defaultItem, caReseted);
    004BFEE3 B105             mov cl,$05
    004BFEE5 8D55F4           lea edx,[ebp-$0c]
    004BFEE8 8BC3             mov eax,ebx
    004BFEEA 8B30             mov esi,[eax]
    004BFEEC FF5674           call dword ptr [esi+$74]
    

    And the “dummy” itself is clean too.

    Spring.Collections.Lists.pas.617: begin
    004C2EC0 53               push ebx
    004C2EC1 8BD8             mov ebx,eax
    Spring.Collections.Lists.pas.618: V := Default(T);
    004C2EC3 33C0             xor eax,eax
    004C2EC5 8903             mov [ebx],eax
    004C2EC7 894304           mov [ebx+$04],eax
    Spring.Collections.Lists.pas.619: end;
    004C2ECA 5B               pop ebx
    004C2ECB C3               ret 
    

    Isn’t it good, Norwegian wood?

  28. Stefan Glienke repo owner

    That was my first thought as well but don’t even need that - just put a class var defaultItem: T into TEnumerableBase<T> and then remove all the local variables for the RSP-20683 workaround and the Default(T) assignment. The class var is guaranteed to be empty - so no extra initialization necessary and stack space plus pro- and epilogue code saved.

  29. Arioch The reporter

    how often my IDE or the compiler die every day

    Well, compiler dies often. So closing and re-opening project became a habbit, gladly Delphi is reasonably fast with it.

    But IDE is much more stable, at least to me. XE2u4hf1 with IDE fixpack only freezes like one a week to me, give or take. Compiler crashes do not propagate to IDE.

    So maybe you’d tweak IDE FixPack and DDevExts, or maybe newer IDE versions got less stable.

    We have one Berlin license, but only one (turf wars in a company with < 50 total personal - pathetic and cruel, when other guys bought XE4 i was begging for them to grab XE2 too, when they 2 years later moved on - they threw me that XE4 license, without XE2).

    And me and one another guy have a legacy project going up from late 1990-s. When i was porting it from D2006 to XE2 it took me months to iron out all the bugs, from component libraries getting belly-up or refactored to old guys C habits, like pointer math assuming Char=Byte…. To pretty gifts from EMBS like breaking XE2 COM interoperability month or two before lanuching XE3.

    So the very idea me coming to boss to demand every year we would be paying to EMBA for at least two licenses, and that for a healthy prospect of porting the project yet again and harvesting vast fields of healthy diverse bugs… does not resonate with me 🙂 “Devil you know”, you see. Even when it tends to destroy string constants 😃

  30. Arioch The reporter

    class var defaultItem: T into TEnumerableBase<T>

    will only benefit one class. Special default-making stub will benefit all clases today and tomorrow.

  31. Arioch The reporter

    Additionally maybe casting Default call outside of this unit and into utils unit will localize ICEs, just not sure WHERE it will localize them 😓

    But in general it seems generics-using code better be split into most tiny pieces possible, or Delphi compiler goes into combinatorial splosion and goes puff. So a special stub type for defaulting feels safer for me.

  32. Stefan Glienke repo owner

    The defect is only in every sub class of TEnumerableBase<T> - also the benefit of not having to pollute the prologue and epilogue with a variable of T (especially if that is a managed type) simply to pass its default value in some case looks very tempting to me.

  33. Arioch The reporter

    yes, stack frame shrinking so much had me shocked,

    and, like i said, i’d better have the “makedefault” safe tool in the toolbox in case i would need it in any other class. But this is your lib, afterall.

    ….and then someone later will optimize the var back into the method :-DDD

    and hopefully Build.exe will see some love too, not only moss-covered XE2 of ages past 🙂

  34. Arioch The reporter

    Tested that RSP. XE2 is affected. Actually, i think i saw similar things like with methodpointer := nil or something along the lines. Finalization maybe. Also only zeroed one pointer of the two. But since nil check also only checked one pointer - that seemed to fit one another, so i took it as “implementation detail”.

    type
      TMethodBummer<T> = class
      public
        procedure Receiver(const M: T; const kind: byte);
        procedure Change(const M: T; const kind: byte);
        procedure Test;
      end;
    
    
    { TMethodBummer<T> }
    
    procedure TMethodBummer<T>.Receiver(const M: T; const kind: byte);
    var P: Pointer;
    begin
      P := @M;
    
      Writeln(Format('Code = %p, Data = %p, 2nd param = %d',
        [TMethod(P^).Code, TMethod(P^).Data, kind]
      ));
    end;
    
    procedure TMethodBummer<T>.Change(const M: T; const kind: byte);
    begin
      Receiver(M, kind);
    end;
    
    procedure TMethodBummer<T>.Test;
    begin
      Change(Default(T), 42);
    end;
    
    ......
    
        with TMethodBummer<TNotifyEvent>.Create do try
          Test;
        finally
          Destroy;
        end;
    
    Project46.dpr.66: Receiver(M, kind);
    004BCDDB FF750C           push dword ptr [ebp+$0c]
    004BCDDE FF7508           push dword ptr [ebp+$08]
    004BCDE1 E80AFFFFFF       call {Project46}TMethodBummer<System.Classes.TNotifyEvent>.Receiver
    Project46.dpr.67: end;
    004BCDE6 5D               pop ebp
    004BCDE7 C20800           ret $0008
    004BCDEA 8BC0             mov eax,eax
    Project46.dpr.71: Change(Default(T), 42);
    004BCDEC B12A             mov cl,$2a
    004BCDEE 33D2             xor edx,edx
    004BCDF0 E8E3FFFFFF       call {Project46}TMethodBummer<System.Classes.TNotifyEvent>.Change
    Project46.dpr.72: end;
    004BCDF5 C3               ret 
    

  35. Arioch The reporter

    ---
     Tests/Source/Spring.TestRunner.pas | 4 ++--
     1 file changed, 2 insertions(+), 2 deletions(-)
    
    diff --git a/Tests/Source/Spring.TestRunner.pas b/Tests/Source/Spring.TestRunner.pas
    index 4c7f27e2..8711d154 100644
    --- a/Tests/Source/Spring.TestRunner.pas
    +++ b/Tests/Source/Spring.TestRunner.pas
    @@ -67,7 +67,7 @@ uses
     {$IFDEF TESTINSIGHT}
       TestInsight.DUnit;
     {$ELSE}
    -  {$IFDEF DELPHI2010}
    +  {$IFDEF DELPHI2010_UP}
       GUITestRunner;
       {$ELSE}
       {$IFDEF MSWINDOWS}
    @@ -197,7 +197,7 @@ begin
       ReportMemoryLeaksOnShutdown := True;
     {$ELSE}
       {$IFDEF MSWINDOWS}
    -  {$IFDEF DELPHI2010}
    +  {$IFDEF DELPHI2010_UP}
       GUITestRunner.RunRegisteredTests;
       {$ELSE}
       if ParamCount > 0 then
    -- 
    

  36. Stefan Glienke repo owner

    Unit tests compiled and run by Build.exe are supposed to run as console.

    The issue with the AV was fixed in master and does not occur in develop due to internal refactorings.

  37. Arioch The reporter

    supposed to run as console.

    If so, then to be consistent

    1. “Console” checkbox should be removed from the Build.exe
    2. Any reference to non-console test runners removed from Spring.TestRunner.pas
    3. Build.exe should report test errors to the developer, installing the library, always. Which would require fixing either Build.exe or the runners or maybe even both.

    Currently it is all the opposite.

    I believe that fixing two lines above and reverting what appears purely accidental suppression of GUIRunner is both proper and easiest possible way.

    Clearly Build.exe does not check return codes and does not display (not even preserve!) test runners output exactly because it assumes them being GUI and thus doing non-avoidable communication with developer on their own. While it proved fragile, it at least makes sense, when no errors are made.

    Otherwise the 3 (or 4) items from the list have to be fixed, which is much more work.

  38. Log in to comment