Team user when removed, doesnot reflect in Branch permission

If we remove a user from the team so that he should not have access to any repo, the branch permissions previously set still shows his name.

Also after removing the user we are not able to modify any branch permission where his name appears, The branch permission has to be deleted and created again.

Big issue when we have 3-4 branch permissions each for 20+ repos...