Occasional 'permission denied'

The last few days, I've sometimes gotten permission errors when pushing changes via ssh/git. Pushing may work fine for several hours, before I suddenly get this error:

Permission denied (publickey).
fatal: Could not read from remote repository.

The problem persists for a while (~30 minutes or an hour), and then everything works fine again. While it lasts, the problem is there for several or all of my repositories. The problem vanishes without any intervention from my side, other than waiting.

I don't commit unusually often or with large change sets. The changes are most often just a few lines, and the problem emerged today with 8 commits within 3 hours.