Clicking the "Remove Connection Or Scratch Org" when a scratch org is selected permanently deletes the scratch org. That is NOT what I would expect to happen. For all other connection types, your org still exists just fine.
Nothing about the messages indicate that the org will be deleted.
This is especially problematic when sharing credentials with your team. They expect “delete” to just remove references to that org. Then it deletes my org with all my work. Not fun.
This also seems to happen with OAUTH connections.
AT A MINIMUM, there need to be BIG BIG WARNINGS if you’re going to leave the delete in. I would argue that a screen that manages CONNECTIONS has no business deleting orgs.