Add capability to show which test is currently running

[Lübbe Onken]

This can be achieved relatively easy by extending the TResultType enum. E.g.:

TResultType = (Passed, Failed, Error, Warning, Skipped, Running);

and adding the neccessary handling, icon and whatever else is necessary to the TestInsight runner.

Then for DUnitX write the following OnExecuteTest handler

procedure TTestInsightLogger.OnExecuteTest(const threadId: Cardinal;
  const Test: ITestInfo);
var
  activeTest: TTestInsightResult;
begin
  activeTest := TTestInsightResult.Create(
    TResultType.Running, test.Fixture.Name + '.' + test.Name);
    activeTest.Duration := 0;
  activeTest.UnitName := test.Fixture.UnitName;
  activeTest.ClassName := test.Fixture.TestClass.ClassName;
  activeTest.MethodName := test.MethodName;
  fClient.PostResult(activeTest);
end;

Same for DUnit

procedure TTestInsightListener.StartTest(test: ITest);
var
  activeTest: TTestInsightResult;
begin
  if IsTestMethod(test) then
  begin
    activeTest := TTestInsightResult.Create(
      TResultType.Running, GetFullQualifiedName(test));
    activeTest.Duration := 0;
    activeTest.UnitName := GetUnitName(test);
    activeTest.ClassName := (test as TObject).ClassName;
    activeTest.MethodName := SplitString(test.Name, '(')[0];
    fClient.PostResult(activeTest);
  end;
end;

Since TResultType.Running doesn't exist yet, I tried it out with TResultType.Warning. Together with SendImmediately := True; It worked out of the box. It's nice to see which test is currently running in a suite of long running tests.

[Stefan Glienke]

I like the idea but do you also have a suggestion on how to display this in the "by type" view?

[Lübbe Onken]

I thought about it while writing the ticket yesterday. I have two suggestions:

1. Make the "active" test a separate tree node, which is always open and always at the top of the tree, no matter which tab is currently active. It would work, but break the "by type" view while the test is running.

2. Like it is now. No separate node, but highlight the "active" test in the tree by using a different icon, bold font, different background colour, whatever. IMO the prettier solution. This would also work in both tabs, but only be visible to the user, if the tree is expanded prior to running the tests. In addition the active test could optionally always scroll into view (I already see the "settings" page grow ;)) unless this causes too much flicker.

[Stefan Glienke]

"I already see the "settings" page grow"

And this is something I want to avoid - I absolutely dislike settings for every tiny piece. I just creates unnecessary overhead for implementing and testing.

The goal of TestInsight is not to provide some full blown GUI runner but to streamline unit testing from within the IDE. Especially for unit tests which always should be as fast as possible and not long running.

I have several thousand test cases that run in just a couple seconds and even turning on SendImmediate almost doubles the time just because it sends each test individually (yes I could do that over named pipes or something so communication is a bit quicker but that is not the point). Sending even more information for each executed test just increases overhead for very little benefit.

Also the approach of using TResultType muddles the design as running clearly is not a test result - it is an additional state. And it might conflict with the filters - do you want to see the running test if all you care about are failed tests (thus hiding all other results)?

[Lübbe Onken]

Same here. I got some test suites with hundreds of tests that execute in a few seconds. SendImmediately and a progress indicator are of no great use there. Others deal with conversions and are I/O bound. My extreme suite at the other end has only ~30 tests. Ten of them run in less than the blink of an eye, the others typically need 0.5 - 3 seconds each. So this suite typcally needs ~30 seconds to execute and it's nice to see the currently active test and the progress being made.

I know that I have abused TResultType and that it is an additional state. I had this idea yesterday and just wanted to see if it would work. For my slow test suites I like this behaviour, for the fast ones it is no benefit. With the current implementation it looks better on the "by type" view than on the "by category" view, since the "active" tests don't jump around in the tree.

Interesting that SendImmediate makes such a huge difference for you. Obviously my test suites are too slow, so that I haven't noticed it yet.

I think I'll put SendImmediately := true/false and the "report active test" methods inside a define like {$IFDEF ReportTestProgress}, so I can turn it on/off in the test projects options and don't have to change the code. This doesn't add an extra setting and it is per project :)

I'm already really happy that I can enable/disable the tests via a GUI. It makes life so much easier. Thanks a lot!

[Lübbe Onken]

Thanks a lot for your work Stefan! Maybe not the propery way of giving you a heads up, but I'm really looking forward to the next release. So much good stuff in there :)

[Stefan Glienke]

Issue #59 was marked as a duplicate of this issue.

[Stefan Glienke]

Also related to #57