I'm having the same issue reported by Robert Hafner on your reviews page (https://marketplace.atlassian.com/apps/1215055/slack-for-confluence?hosting=cloud&tab=reviews) "Changes made to the configuration after the initial configuration are silently discarded and ignored."
I first added a slack channel to an entire space. This was creating too much noise, so I blanked out the channel name in the settings and saved it. After that the notifications stopped.
Then when I added a different channel to a specific page, the original channel was sent the updates, instead of the new one. I've search everywhere in the settings for the original channel name, but it isn't in the settings anywhere. So posts are being sent to the wrong channel, but only on updates to the single page I added it to.