Delete emails from GUI

[Cassio Simões]

Couldn't we have a "With Selected:" delete button on the GUI for auditors with delete permission?

Sometimes, for example, there are some SPAM emails that need removal (that goes through the mail server filter) and also email with big attachments from past import, or even emails that are confidential and must be completely erased

Not every IT administrator (like myself) have enough SQL skills and delete emails becomes very complicated...

[Janos SUTO]

Well, such a button is definitely possible, however there are some other (non technical) aspects against such a button. I believe that such delete feature would undermine the purpose of an email archive, that is being a secure and reliable message store if someone could just remove message on arbitrary conditions, other than the company's message retention policy. Anyway I'll post this issue on the mailing list, and let other piler users share their opinion on the matter. Feel free to join the list.

[eXtremeSHOK]

Deleting of emails should be for the super admin only and Only when an entire domain (example.com) is removed from the archive, their should be an option to have those emails removed. However, a record of exactly what was removed (message id, etc) should be saved to a log.

Depending on the datasize, i would say headers should be kept as a record.

As for deleting, NO EMAILS should ever be able to be removed from the archive as long as its an archive for the domain. Rather allow the auditor to hide emails. Then one could flag the emails as hidden, they would show up if a user has permission to view hidden emails.

Irrespective of user permissions, the system would need to show that there are hidden emails. eg. Show the header but place a message for the content: this email was hidden by auditor X, please contact admin for permission to view emails.

As soon as emails are deleted from the archive, the archive would loose legal and regulatory compliance.

Pretty much piler would become useless.

[eXtremeSHOK]

Off topic: No person should have ssh or root access to an archive server. This is why configuration management exists everything should be documented and audit trails followed.

I will assume your archives do not need to fall under legal requirements, which require you to prove the archive has not been manipulated and/or cherry picked.

If emails can be deleted from the webinterface, someone that gains access could purge all or some of the emails or data. This poses a far more serious risk than just unauthorized access.

My point is.. if you allow people to delete items from the archive, it is no longer a valid for legal and regulatory requirements.

So if the feature is enabled via the config ( allow_deletes = true; ), warnings would need to be placed to show users the archive is not "complete" and no longer complies.

[eXtremeSHOK]

Its off topic: if you need to know more contact me. All our servers we manage are 100% automated. Maintenance, updates, etc. If you employ proper usage of configuration management and write idempotent scripts everything can be automated. No admin has access to the servers. This is exactly how large companies manage 10 000's of servers.

Since admins do not have access, there is no risk of emails being deleted and every maintenance task has an audit log.

[eXtremeSHOK]

Example: if the feature is enabled via the config ( allow_deletes = true; ), warnings would need to be placed to show users the archive is not "complete" and no longer complies. This would also enable the delete functions.

[Janos SUTO]

OK, then, it's possible to make it. I'll let you know when it's ready to test.

[Janos SUTO]

I've started working on it. A question: is it OK to set the retention time of the selected emails to the past, and let the daily purging job do the job? In this case we don't need a separate removal utility with some redundant functions.

[Janos SUTO]

I applied yet another config.php variable (ENABLE_DELETE), and I suggest to fabricate a text to the login screen explaining that the delete feature is enabled with all its consequences.

I also think about to add some visual effect to the search screen, eg. a different background colour for the search field, whatever.

[Cassio Simões]

I think the ideal would have a "delete" button on the GUI, so deleting future (wrong dated) and unwanted emails could be easy...

Regards

Cassio

[Janos SUTO]

You'll get the delete button for sure. The question is that if it's ok if the delete button only marks the selected messages ready for removal, and then the daily purge cron job finishes the job. This should be the most straightforward approach.

[Janos SUTO]

OK, just pushed the code to the master branch to add this feature (for auditors only). Note that it doesn't actually removes anything just sets the retained field to the current timestamp, and hides it from the search hits, until the daily purge job removes the selected messages.

To enable it, add the following to config-site.php:

$config['ENABLE_DELETE'] = 1;

[Cassio Simões]

I see no problem at all, it's great that way.

[eXtremeSHOK]

You might as well add support for restoring accidentally deleted emails.

Basically show a list of all mails marked for deletion, and allow the user to "restore/un-delete them". Ie. Remove the delete flag of the message

Delete jobs should only remove emails 24-72 hours after the actual mails were marked for deletion.

[eXtremeSHOK]

You might as well add support for restoring accidentally deleted emails.

Basically show a list of all mails marked for deletion, and allow the user to "restore/un-delete them". Ie. Remove the delete flag of the message

Delete jobs should only remove emails 24-72 hours after the actual mails were marked for deletion.

[Janos SUTO]

This feature is possible, though it complicates things. Currently I change the retained column only, the deleted column is set by pilerpurge when the message is actually purged, and thus no turning back. Unfortunately it doesn't wait 24-72 hours. Normally it's a daily job, though you are free to run it once a week, etc. However after changing the retained column there's no way to revert its value.

Surely, it can be done introducing an auxiliary table, but I'm not sure if it's worth the hassle to provide an undelete feature.

[eXtremeSHOK]

You don't need to add a table , add a row called deletestatus, set that to 1 for delete 0 as default

[eXtremeSHOK]

Correction

Add a column "delete" default it to null

Then set a timestamp for the specific email.

Pretty much any email with the delete row not null is a marked for deletion email.

Your piler purge can do a simple MySQL query for older than current timestamp + 24hours to delete those emails.

This ensures that a minimum of 24 hours must pass before it is purged.

The null default value won't increase size of the table.

[Janos SUTO]

There's already a column called "deleted" :-) Anyway it may be true that a default null column doesn't occupy much space, however for existing installations with lots of data/rows twisting the table scheme may be painful, so I'd try to avoid it if possible. I'd say let the topic opener test the feature as it is at the moment, then we may fix later it if needed.

[Janos SUTO]

OK, I consider the feature done. Test it, and let me know if this is what you had in mind.