Piler showing only a few recipients even though Exchange Envelope has nearly 50

[Edwin Nichols]

We noticed today that an email to a distribution list that contained all recipients in the company only indexed for a single employee, along with the distribution list and one BCC recipient added by Exchange to the list as part of a transport rule. We journal to 2 archiving solutions currently, and the other solution correctly indexed all recipients, so we know that Exchange sent the full list in the envelope. I performed a test with a smaller distribution list with the same results. The message in Piler shows the distribution list and a carbon copied recipient, but does not show the members of the distribution list. The other archiving solution shows all members. I can provide examples directly, but since it contains email addresses and other confidential information, do not want to post here.

[Janos SUTO]

Yes, I'd like to see a sample, please send one to me.

[Janos SUTO]

Bitbucket is not really a mailing platform, so could you send it to me as a usual email to my email address? Perhaps as a zipped attachment.

[Janos SUTO]

I've checked the messages you sent me, and I believe that piler works properly, since it displays the recipients as they are in the header, ie showing the distribution list, and not expanding it.

Note that this is merely an implementation issue, since piler uses a different approach than mailstore. Piler expands the membership info internally on the fly when the user uses the gui. You may check it by viewing the settings page.

[Edwin Nichols]

Hi Janos,

It seems that real-time checking of distribution list memberships could lead to unexpected results, and does not fully match the sample sent. In each sample, there was a single member of the distribution list that was included in Piler’s indexing, but none of the others. This user is not explicitly in the header (Antzinas), so there was at least a beginning of parsing members during the initial indexing by Piler unless I am missing something. Also, if I understand correctly what you are saying is done, if an employee moves between departments, they will seem as if they received an email sent to a distribution list in the past even if they were not a part of that list at the time.

For example,

Email sent to “Main Office” distribution list on January 1, 2016. At the time, Employee A works from home and did not get the email sent to “Main Office”.

On March 1, 2016, Employee A starts working from the main office location, so becomes a member of the distribution list “Main Office” at that time.

On April 1, 2016, Employee A searches for emails and sees the email from January 1, 2016. They never received it, but it will show in their search results as they are a member of the list since March 1.

We are using Piler to index emails and our intent is to have a legal record that shows an accurate reflection of what was sent/received per employee, whether they later became a member of a list or not. Without Piler indexing the current members of the distribution list at the time the message is archived, the results will never be accurate, even if the employee didn’t work here during the time the message was sent to the distribution list.

If I am misunderstanding what you indicated, please let me know.

Edwin Nichols 

[Janos SUTO]

I see your point, let me think about it and find a solution.

[Janos SUTO]

I think I've managed to fix it. I suggest you to try it in a test environment: install the master branch, and see if pilertest /path/to/message.eml shows the correct recipients.

Note that I've kept the distribution address among the recipients, so the result is something like

from: *somebody1 user1 somecompany [e-mail] user1 somebody1 somedomain com  (somedomain.com)*
to: *somebody2 stephen somecompany [e-mail] stephen somebody2 somedomain com system development [e-mail] listaddress somedomain com [e-mail] jack aaaaaa somedomain com [e-mail] katherine bbbbb somedomain com [e-mail] stephen ccccc somedomain com [e-mail] jerry ddddd somedomain com list address  (somedomain.com )*

[Janos SUTO]

I assume you would accept the fix, hence closing the issue.