Replying to issue emails doesn't work

When receiving emails from the issue tracker, it's not possible to reply via email to close the ticket despite the line:
"--- you can reply above this line ---"

Please fix or explain how to configure the settings better.