If print-level and print-length are set to small values, then the achievements-file will end up being corrupted.
In my particular case, print-length was not big enough to contain the object when the achievements were saved. On the next load this gives the error: "emacs-achievement-name accessing a non-emacs-achievement". If you open the achievement file, you see that the last item is ...
The fix is:
(defun achievements-save-achievements () "Saves achievements to a super secret file." (interactive) (let ((print-level nil) (print-length nil)) (with-temp-file achievements-file (prin1 achievements-list (current-buffer)))))