Feature: Create continuous feedback loop mode.

[Justin Julicher]

With most classes we are able to go to the related Test class using the related files function.

It would be good if we could use this information to run those tests every time a successful save is done for an apex class.

This way we can get the feedback straight away if the changes break Unit Tests.

I'm not sure how this would affect the limitations on test invocations ( I seem to remember somewhere there were maybe 500 or 1000 invocations allowed per day).

[Scott Wells]

You pretty much stated my main concern about this type of feature...exhausting the 24-hour total API request limit. That limit is 15K in Developer Edition orgs. Curiously it's showing up as 5M in scratch orgs! That seems suspicious to me, but perhaps it's not.

I'll think about this...there are things I do genuinely like it about...I'm just not sure it's compatible with the Apex governor-limited Development-as-a-Service model.

Thanks for filing!

[Justin Julicher]

Thanks Scott,

FYI For sandbox orgs it's 500 or 20 * number of Test classes you have....

To be honest I think this would be fine if your the only developer working in the org... in a 9 hour day that would work out at roughly a save every minute... I don't know about you but with meetings and other distractions around the office I would find this really hard to hit most days....

If you were to implement this; assuming you can read the limit somehow I would set auto run tests to stop at 80% of the limit giving the user a bit of flexibility...

In Scratch orgs, if it is indeed 5M, then there's no need for a restriction there at all.

[Justin Julicher]

Hi Scott - any more thoughts on this?

I was thinking if implemented it would be good to give a warning if say 25% of tests runs have been executed - disable for 24 hours? Can’t remember if this is a rolling limit or resets daily. So you could adjust it depending on the way they are calculated.

‌

‌

[Scott Wells]

Yeah, I was actually noodling on this after running across it while review some older issues. I have a few thoughts and have added it to my near-term consideration list. I'll let you know more once I've had a chance to play with it a bit.

[Attila Hajdrik]

Scott, do you have any update on this?

[Scott Wells]

Actually, yes. I'm starting to collect unit testing enhancements for an upcoming update, and right now this would be on the list along with a few other related items. I'm not 100% sure when that will happen, but hopefully in the next few builds. I'll post a more specific update here once things start rolling.

[Scott Wells]

Issue #2408 was marked as a duplicate of this issue.

[Scott Wells]

With the recent enhancements to unit testing and code coverage in 2.3.1.1, 2.3.1.2, and 2.3.1.3 (also demonstrated extensively in a demo video) I think the desired functionality here is effectively delivered. It may not be in direct response to small-scale deployments such as those from deploy-on-save, but between the new “Dependents” and “All Tests”/”Changed Only” run configuration types, it’s now possible to execute tests and measure coverage for the desired/applicable scope of local work.

[Scott Wells]

Resolving based on the previous statement, but if those assertions don't hold up in reality, please feel free to reopen with a clear description of the remaining gap.