Control Freak doesn't log Override Yes/No changes to audit log

[Marijn van Zon]

Hi,

Thank you so much for the update where Control Freak now logs all changes to the audit log. This really helps us a lot!

During testing everything looked great, except for that changing Override to/from Yes/No was not logged to the Audit log. This creates a caveat. For example take the following situation:

1. In the repository ABC configuration set Override to No for General Policy.
2. In the repository ABC configuration enable General Policy > Author must be a valid Bitbucket User.
3. In the global configuration enable General Policy > Author must be a valid Bitbucket User.
4. In the repository ABC configuration set Override to Yes for General Policy (but do not change any values).
5. In the global configuration disable General Policy > Author must be a valid Bitbucket User.

What you will see in the audit log now is:

1. In the audit log

   1. an entry is added for any differences in configuration between global (or project) and the repository ABC.
   2. no entry is added that Override was changed from Yes to No_._

2. In the audit log an entry is added: PROJ/repo | {"changed":{"authorValidate":"yes"}}

3. In the audit log

   1. an entry is added: GLOBAL | {"changed":{"authorValidate":"yes"}}
   2. no entry is added for individual projects or repositories that have Override so to No and thus were affected by this change

4. In the audit log no entry is added that reflects changing Override from No to Yes

5. In the audit log

   1. an entry is added: GLOBAL | {"changed":{"authorValidate":"no"}}
   2. no entry is added for individual projects or repositories that have Override so to No and thus were affected by this change

The result is:

• After step 2 for repository ABC Author must be a valid Bitbucket User is enabled.
• After step 5 for repository ABC it is disabled.

Now if you were to follow the same steps, but skip step 4 the result is:

• After step 2 for repository ABC Author must be a valid Bitbucket User is enabled.
• After step 5 for repository ABC it is still enabled.

But the audit will be identical whether you skip step 4 or not. It’s impossible to deduce for repository ABC when Author must be a valid Bitbucket User was enabled and when it was disabled.

I assume there are two ways to resolve this.

A. Add an entry in the audit log for 1.b and 4 so that any changes in Override are also reflected in the audit log. This will remove the need to log the differences when changing the value of Override. The entries from 1.a can then be removed.

B. Add entries for differences between global vs project vs repository like is done with in 1.a, just not only when Override is changed, but also when a higher configuration is changed to propagate the audit entries for changes to project and repository level. So for example step 2 would then add not one, but three entries in the audit log:

GLOBAL | {"changed":{"authorValidate":"yes"}}
PROJ | {"changed":{"authorValidate":"yes"}}
PROJ/repo | {"changed":{"authorValidate":"yes"}}

Option A is I think a must-have for completeness and from an audit perspective will solve the problem.

Option B is optional and for me a nice to have. It might create some clutter, but it will also be easier to read the logs because if something is changed you don’t have to parse the logs (perhaps going back for months) to figure out if where Override was Yes and where it was No.

Regards,

Marijn van Zon.

Edit: updated description and steps for better clarity

[Marijn van Zon]

‌

‌

[Julius Davies [bit-booster.com]]

Thanks for an amazing and detailed bug report!

v2020.02.07 now includes logging of the overrides.

Current implementation also logs the net result, and so if “override” is changed to “no” and that also happens to cause some changes to the net values, those also get logged at that moment.

I don’t think option B can work for the way Control Freak is currently built. Firing off audit entries for all downstream projects and repositories when higher-level edits happen just doesn’t suit our current architecture. We would have to access every repository to do that (on every edit!)! Some customers have 10,000+ repositories.

‌

BELOW IDEA FOR FURTHER IMPROVEMENT - WHAT DO YOU THINK…?

I’m thinking we should probably also log the values (that didn’t change) when “override” is set to “yes”. Maybe something like this:

{“added”: {“override_general”, “yes”}, “changed”: {“some_key”, “changed_value”}, “unchanged”:{“another_key”, “unchanged_value”}}

And so you could look inside this “unchanged” entry to see what “overridden” values were left alone (but basically locked-in by the override).

‌

‌

‌

[Marijn van Zon]

Hi,

Sorry for the late response. Good point that with that many respositories option B isn’t feasible. For me it’s fine to only have option A as you have it now :). Thank you for the fix!

Adding the “unchanged” ones might be nice so you can at least you that point in time as a starting point if you need to analyse the logs to figure out what the setting was at a given point in time. But as I mentioned before, just logging the override change like is done now was the most important thing.

Regards,

Marijn van Zon.

[Julius Davies [bit-booster.com]]

Thanks Marijn ! Glad to hear things are good now. I’m going to leave this ticket open for now until we implement the final proposed improvement (logging unchanged items when overrides change).