1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

Get mailing list members from Active Directory

Issue #679 closed
Thoth created an issue

Hello,

i have piler 1.1.1 up and running fine. One last thing is open: When a mailing list ist used as a to-adress, piler stores the mail as if this list is a real user. If i create a user account with the name of the list, i can login and see the mail. But the members of the list can't see the mail they received.

In the documentation i found the following paragraph: "... piler will hide an email from a user if he was (only) in the Bcc: field. This limitation has another side effect related to external mailing lists. You have to maintain which user belongs to which external mailing lists, otherwise users won't see these messages. Internal mailing lists are not a problem as long as piler can extract the membership information from openldap or Active Directory."

Well, i have an Active Directory and the list is internal so my question is: how do i do that? How can i persuade piler to look for the members of an mailing list in the active directory and store the mails for them?

(BTW: I know the X-Envelope-To Header is better suited for this problem, unfortunately our mailserver doesn't provide this.)

Many thanks.

Comments (18)

  1. Janos SUTO repo owner

    You experience such problem, because the gui is not aware of that which user belongs to the internal mailing list. The solution is not creating a user for the mailing list. Rather you should configure the group resolution to make it right. So edit config-site.php, and add the following entries (I assume you have properly setup ldap authentication):

    $config['LDAP_ACCOUNT_OBJECTCLASS'] = 'user'; $config['LDAP_DISTRIBUTIONLIST_OBJECTCLASS'] = 'group'; $config['LDAP_DISTRIBUTIONLIST_ATTR'] = 'member'; $config['LDAP_MAIL_ATTR'] = 'proxyAddresses';

  2. Thoth reporter

    Thank you for your answer. Unfortunately it made things not better: now i can't login at all.

    Here are the LDAP-lines from my config-site.php

    #########################
# LDAP am Active Directory
##########################
$config['ENABLE_LDAP_AUTH'] = 1;
$config['LDAP_HOST'] = 'ldap';
$config['LDAP_HELPER_DN'] = 'CN=LDAP Abfrager,OU=***,DC=***';
$config['LDAP_HELPER_PASSWORD'] = 'xxxx';
$config['LDAP_AUDITOR_DN'] = '';
$config['LDAP_BASE_DN'] = 'DC=***';
$config['LDAP_ACCOUNT_OBJECTCLASS'] = 'user';
$config['LDAP_DISTRIBUTIONLIST_OBJECTCLASS'] = 'group';
$config['LDAP_DISTRIBUTIONLIST_ATTR'] = 'member';

# This works, i can login with the E-Mail-Address and the ADS-Password
#$config['LDAP_MAIL_ATTR'] = 'mail';

# With this, i can't' login with the E-Mail-Address and the ADS-Password
$config['LDAP_MAIL_ATTR'] = 'proxyAddresses';
###################

    If I trace the LDAP-Dialog with Wireshark, I see the following filter when "$config['LDAP_MAIL_ATTR'] = 'mail'; " is active:

    Filter: (|(|(&(objectClass=user)(mail=MyName@example.com))(&(objectClass=group)(member=MyName@example.com)))(&(objectClass=group)(member=CN=,OU=,DC=***)))

    With "$config['LDAP_MAIL_ATTR'] = 'proxyAddresses'; " I see this filter:

    Filter: (&(objectClass=user)(proxyAddresses=smtp:MyName@example.com))

    Just to be sure, that i'm not doing something basically wrong: I can create disttribution groups in Active Directory and give them an E-Mail-Address. But as member i can only assign users (which have an E-Mail-Address as one of their properties) and no E-Mail-Adresses. We don't have an Exchange Server, so i don't know, if this is enough.

    Sorry for the long text.

  3. Janos SUTO repo owner

    First, you have to create a helper ldap account for piler. I'm not sure if "CN=LDAP Abfrager,..." is such an account, and make sure you can login with it. Then check if the "(&(objectClass=user)(proxyAddresses=smtp:MyName@example.com))" filter can actually return something. You may check the mail log as well, the gui syslogs.

    You mention that you don't have exchange. Then try setting

    $config['LDAP_MAIL_ATTR'] = 'mail';

    And the next time, please start with describing what you have. It's much more easier to help then.

  4. Thoth reporter

    I'm sorry for being too short but I thought that this is treated as looking for instructions (you need this and then do this, do that etc.) and and not as an technical problem. Here comes the whole story:

    We use mailing lists which are configured on our mailserver (which s not an Exchange server). Our server doesn't provide an ""X-Envelope-to"-header, so a mail to the members of the List "MailingListName@example.com" a Mail has this header:

    Return-Path: HomepageAdmin@example.com
From: Script HomePageMail <HomepageAdmin@example.com>
Sender: HomepageAdmin@example.com
Reply-to: HomepageAdmin@example.com
To: MailingListName@example.com
X-Mailer: yyyy
Message-ID: <ID>
Subject: ....
Content-Transfer-Encoding: 8BIT
Content-Type: text/plain; charset=ISO-8859-1

    Our mailserver mirrors all Mails it gets to the piler archive.

    I'm a member of this list and receive this mail. I use LDAP to authenticate the GUI-login against our Active Directory and this works. But I can't find the mail in my archive. I only see Mails which were sent directly to me. Just for testing i created an user "Test" with the Address "MailingListName@example.com" as its Mailaddress in our Active Directory. If i use this account data in the GUI i can see the mail. So piler stores the mail for an user "MailingListName@example.com" but not for me as a member of the list.

    I did some googeling to find a solution and found the cited text from the documentation. This seemed to fit, so i asked how to do that. I was assuming that piler then checks the to-address againt our Active Directory, sees that it is a distribution list and then retrieves the members.

    To be clear: I was not asking how i can setup the GUI-login to use LDAP with an Active Directory because that works already.

    To your questions:

    • Yes, "CN=LDAP Abfrager,..." is the helper account and piler can login with it.

    • Our server answers to an question with the filter "(&(objectClass=user)(proxyAddresses=smtp:MyName@example.com))" with zero results. This is consistent with the log which says "0 hits":

    Apr 7 10:23:15 archiv01 piler-webui[30946]: ldap query: base dn='DC=***', filter='(&(objectClass=user)(proxyAddresses=smtp:smtp:MyName@example.com))', attr='', 0 hits

    • GUI-Login works if i use $config['LDAP_MAIL_ATTR'] = 'mail';
  5. Janos SUTO repo owner

    The issue boils down to assigning the appropriate email address list to the given user. The gui should be able to query all of his email addresses including the aliases, and even the distribution list memberships (=mailing list addresses).

    After a successful login, click on the settings (top right corner), and verify that.

  6. Thoth reporter

    Thank you for your answer. I suspect now, what the reason is. I hope i describe it understandable as english isn't my native language. So:

    If I login with my name the settings show only my normal address, no list. You said, that the LDAP query should get the list addresses, so i looked closer at the LDAP dialog between piler and our server. There are in fact two LDAP querys: the first with the helper account for my data. Piler uses the filter

    (&(objectClass=user)(mail=MyName@example.com))

    and our server answers with the DN

    CN=Lastname\, Forename,OU=...,DC=...

    Please note the masked comma, because our account naming scheme is "Lastname-Comma-Space-Forename".

    As second query piler uses the information to bind with my name and asks for the e-mail-addresses. It uses the filter

    (|(|(&(objectClass=user)(mail=MyName@example.com))(&(objectClass=group)(member=MyName@example.com)))(&(objectClass=group)(member=CN=Lastname, Forename,OU=...,DC=...)))

    which our server answers with zero results.

    Please note: now the comma is unmasked.

    Crosscheck: I took an Active Directory account with the name "sbe" and the e-mail adress sbe@example.com as its property. Then i made this account to a member of the active directory distribution list MailingListName@example.com.

    When i logged in in the gui with this account and checked the settings i saw two addresses: sbe@example.com and MailingListName@example.com. This works.

    So I'd say that the reason is the unmasked comma in the second LDAP query or rather the fact that Piler seems to delete the escape character it got in the process to form the filter that it uses to query the addresses.

  7. Janos SUTO repo owner

    Thanks for the clarification. Can you show me the 'ldap auth against' line from the maillog when you login? If it has the \, sequence, then try the following:

    edit model/user/auth.php, and locate the following line (~215th), and comment it out:

    $a['dn'] = stripslashes($a['dn']);

    Then try to login, and check the settings again. Btw. what version of AD / windows server do you have?

  8. Thoth reporter

    The Windows is a Windows Server 2003 with the Active Directory being in Windows Server 2003 Mode. Here are the lines from /var/log/mail.log

    Apr  8 11:35:13 kassrvarchiv01 piler-webui[7430]: ldap query: base dn='DC=...', filter='(&(objectClass=user)(mail=MyName@example.com))', attr='', 1 hits
Apr  8 11:35:13 kassrvarchiv01 piler-webui[7430]: ldap auth against 'ldap', dn: 'CN=Lastname\, Forename,OU=...,DC=...', result: 1
Apr  8 11:35:13 kassrvarchiv01 piler-webui[7430]: ldap query: base dn='DC=kas', filter='(|(&(objectClass=user)(mail=MyName@example.com))(&(objectClass=group)(member=MyName@example.com))(&(objectClass=group)(member=CN=Lastname, Forename,OU=...,DC=...)))', attr='', 1 hits
Apr  8 11:35:13 kassrvarchiv01 piler-webui[7430]: username=MyName@example.com, event='logged in'

    "ldap auth against" shows the "\". If i comment the line with the stripslash-command out (it's line 200) the log shows:

    Apr  8 11:47:06 kassrvarchiv01 piler-webui[8122]: ldap query: base dn='DC=kas', filter='(&(objectClass=user)(mail=MyName@example.com))', attr='', 1 hits
Apr  8 11:47:06 kassrvarchiv01 piler-webui[8122]: ldap auth against 'ldap', dn: 'CN=Lastname\, Forename,OU...,DC=...', result: 1
Apr  8 11:47:06 kassrvarchiv01 piler-webui[8122]: ldap query: base dn='DC=kas', filter='(|(&(objectClass=user)(mail=MyName@example.com))(&(objectClass=group)(member=MyName@example.com))(&(objectClass=group)(member=CN=Lastname\, Forename,OU=...,DC=...)))', attr='', 0 hits
Apr  8 11:47:06 kassrvarchiv01 piler-webui[8122]: username=MyName@example.com, event='logged in'

    No the settings show no address.

    I found a strange thing (at least for me) if I look a the LDAP-Dialog between Piler and the server. The normal dialog goes as follows:

    bindRequest "CN=..." (Helper DN)
    bindResponse success
searchRequest with filter (&(objectClass=user)(mail=MyName@example.com))
    searchResEntry "CN=Lastname, Forename,OU=...."
bindRequest "CN=Lastname, Forename,OU=...."    
    bindResponse success    
searchRequest with filter (|(&(objectClass=user)(mail=MyName@example.com))(&(objectClass=group)(member=MyName@example.com))(&(objectClass=group)(member=CN=Lastname, Forename,OU=...,DC=...))) 
    searchResEntry "CN=Lastname, Forename,OU=...." (no hits)
unbindRequest
unbindRequest

    In the settings only my Mail-address is shown.

    Now, after i commented that line out the dialog shows:

    bindRequest "CN=..." (Helper DN)
    bindResponse success
searchRequest with filter (&(objectClass=user)(mail=MyName@example.com))
    searchResEntry "CN=Lastname, Forename,OU=...."
bindRequest "CN=Lastname, Forename,OU=...."    
    bindResponse success    
unbindRequest
unbindRequest

    The second search is completely missing. This explains why the settings the gui show now no mail-address at all. But i can search for my mails if i klick on the search button.

  9. Janos SUTO repo owner

    OK, so we need the stripslashes stuff. What I cannot see from here, how to fix the search query to get your distribution lists?

  10. Thoth reporter

    Yes, it seems that the query-fuction stumbles upon seeing slashes. Why else should it need stripslashes?

    I made two more tests: First i renamed my account in the active directory to "Fornename Lastname" and it worked. Unfortunately this is not an option for the whole dirctory as we use it to search for e-mail address of recipients in the case the user doesn't know it. For this purpuse the list has to be presented with the lastname first because then it is correctly sortet. (And the users mostly know the Lastname but rarely the Forename - but that's another story)

    For the second test i put a * in the name ("Forename* Lastname") to see whether this would be masked. To be short: it wasn't.

    This prooves that the query-function doesn't auto-mask any special character - which it should because strings with slashes are not accepted.

    Maybe there is an option or a parameter which tells it to auto-mask?

    (How do others come around this? The use of the scheme "lastname comma space forename" is not that uncommon)

  11. Janos SUTO repo owner

    I don't remember if it was a problem in the past. However it's unusual to me that when stripslashes() enabled you get a single hit back. Usually people are part of several groups. Anyway I'd like you to verify what result you get, because it seems that it doesn't contribute to the email addresses you have.

  12. Thoth reporter

    Well, the account is only member of one active directory distribution group (and member of many security groups btw) as i didn't need active directory distribution groups (our mailserver has its own lists). If I make it a member of a second distribution group i see three adresses on the settings page: my own adress and the adresses of the lists. The LDAP dialog shows three server responses to the gui query, so that is consistent. If it works, it worksright

    I don't know anything from PHP, but from what i do (not) see in the LDAP dialog, i'd guess, that the query is not executed if the LADP string contains characters which need to be masked.

    I'll do some more testing and report the results.

  13. Janos SUTO repo owner

    If you can see the address of the 2nd distribution group, then it must work properly.

  14. Thoth reporter

    I think, the discussion goes a little bit off track. To be clear:

    The gui works and shows all groups, if and only if the DN of the user has no comma in it. The DN has to be "CN=Forename Lastname,CN=Users,DC=...".

    If the DN has a comma in it, like "CN=Lastname\, Forename,CN=Users,DC=..." it works not. This is the scheme we use. If the DN uses this scheme, there are no addresses from the distribution lists shown in the gui on the page settings. This is, because the stripslashes-function removes the "\" in the dn, so that Piler asks the Active Directory for (&(objectClass=group)(member=CN=Lastname, Forename,OU=...,DC=...))) in the filter and this is not allowed for an Active Directory LDAP filter. Due to this, this filter returns no results.

    If i force the filterstring to have the "\" again (commenting out the stripslashes command, use of str_replace(", ","\, ", $a['dn']) etc.) the query isn't executeted as i can see in a network trace where i trace the ldap packets that are exchanged between Piler and our server. So there are no results shown on the settings page.

    In this case (stripshlashes commented out) i can find some messages in /var/log/apache2/error.log:

    [Mon Apr 11 13:36:32.932621 2016] [:error] [pid 3059] [client 192.168.100.122:53182] PHP Warning:  ldap_search(): Search: Bad search filter in /var/www/piler/system/database/ldap.php on line 35, referer: http://pilerserver/login.php
[Mon Apr 11 13:36:32.932715 2016] [:error] [pid 3059] [client 192.168.100.122:53182] PHP Warning:  ldap_get_entries() expects parameter 2 to be resource, boolean given in /var/www/piler/system/database/ldap.php on line 37, referer: http://pilerserver/login.php

    This explains why the query isn't executetd but why does the query do this? The filter should be Active Directory compliant as the string came frome the Active Directory and remains untouched.

  15. Thoth reporter

    Got it! After heavy use of Google I found this link:

    https://krivokuca.net/2012/08/word-of-advice-escaping-ldap-filter-values-in-php/

    There is described, that the backslash has to be coded in a special way. So I added the following function to /var/www/piler/model/user/auth.php

     public static function escapeLdapFilter($str = '') {
        // The characters that need to be escape.
        //
        // NOTE: It’s important that the slash is the first character replaced.
        // Otherwise the slash added by other replacements will then be
        // replaced as well, resulted in double-escaping all characters
        // replaced before the slashes were replaced.
        //
        $metaChars = array(
        chr(0x5c), // \
        chr(0x2a), // *
        chr(0x28), // (
        chr(0x29), // )
        chr(0x00) // NUL
        );
        // Build the list of the escaped versions of those characters.
        $quotedMetaChars = array ();
        foreach ($metaChars as $key => $value) {
        $quotedMetaChars[$key] = '\\' .
        str_pad(dechex(ord($value)), 2, '0', STR_PAD_LEFT);
        }
        // Make all the necessary replacements in the input string and return
        // the result.
        return str_replace($metaChars, $quotedMetaChars, $str);
    }

    Next I replaced the line

    $a['dn'] = stripslashes($a['dn']);

    with

    $a['dn'] = $this->escapeLdapFilter($a['dn']);

    and it works. Now I can see also the addresses of the distribution groups in which i am a member, no matter what the DN looks like.

    Please allow me to add some thoughts:

    • I think the function is also needed in the case the Single-Sign-On is used, but i didn't test it.

    • The fact, that the group membership is looked up when the user logs in to his archive has two side effects:

    1. Imagine, a user gets a new group membership. In this case he will also see all mails that were sent to the group before he became a member.
    2. The opposite when a user looses a membership: The he will not longer see the mails he received when he was a member.

    What does an auditor see in these cases?

    Thank you.

  19. Log in to comment
Assignee
Type
task
Priority
minor
Status
closed
Votes
0
Watchers
2
Jira: the preferred issue tracker for Bitbucket. Join the team!