search and expert search filters show always all user emails with IE

[Urs Frei]

tested with IE8,9,10,11 and piler 0.1.25-rc2 and master branch eb334317ca69 tested from: to: a: in the search field and in expert search From, To and Date. messages in the apache error log at the same time: Undefined index: searchtype in helper.php on line 45 Undefined index: sort in helper.php on line 80 Undefined index: order in helper.php on line 81 In expert search the attachment filter with the checkboxes works with the scenario: F5 -> select checkbox -> search, F5 -> change checkbox selection -> search. Without F5 i get all user emails again and helper.php messages in the error log. No any search problem with Firefox so the sphinx search is working. As i can see in the mail log the sphinx get no any additional search filters with IE exept attachment checkbox selection after F5.

[Janos SUTO]

As what user are you logged in? And what's the search query you can see in the maillog? Is it the same in case of IE and FF? Could you show me a screenshot with the problem? By using IE 11 I couldn't reproduce the php warnings you mentioned, though I'm a bit lost trying in the process.

[Urs Frei]

Logged in as normal user no admin or auditor! FF no any problem. I did a little playing with search and find out the search and expert search is working with every IE too but i have to press the button "search" two times always. Scenario: search field input e.g. from: sj* -> on first click on search = all my emails and maillog output: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH(' (@ from urs.freiXmydomainXcom| ......... -> second click = emails from sj* to me only and mail.log output: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE MATCH('( @ from sj* ) & (@ from urs.freiXmydomainXcom| ....... Same with expert search exept one step more to open the expertsearchbox again of course. And i have the same effect after a search if i open a email from the list the first time. First click = invalid id: , second click the email opens. Maybe its a IE problem with some functions in piler-in.js.

[Janos SUTO]

If you can access others' emails as an ordinary user, then it's a major security issue. I'd like you to check if on the demo page as well. (use

http://piler.aaa.fu.acts.hu/

and use the following account:

[e-mail]:ubuntu

Any using IE 11, I could not reproduce the push the search button twice symptom.

[Urs Frei]

sorry for my bad english, i can't never see emails from other users just my emails. What i mean is that my search filter input isn't arrived in the sphinx search string on first push of the search button. i can see it about the given total number of emails in the status. I took the appliance for the installation and switched to apache2 only. Next step is a fresh installation from the real basic. I will inform you.

[Janos SUTO]

OK, let me know if there's an update.

[Urs Frei]

sorry for the late reply. I had tested with a new installation from scratch with the same result. Then I turned off the SSO only and the IE problem was gone. I can reproduce it on two different installations, so could you test it with SSO on your site too but keep in mind it happens only together with IE not with FF.

[Janos SUTO]

OK, thanks for the feedback, I'll try it as well.

[Urs Frei]

I can imagine that you are very busy but I want to ask you whether there are any news about?

[Janos SUTO]

Yes, I managed to reproduce the issue with IE. Unfortunately it appears that IE is not aware of the presence of the select fields when using SSO. More investigation is required, though, so please give some more time.

[Urs Frei]

Ok, good to know, i will wait of course.

[Janos SUTO]

I can confirm the issue. I have experienced it even after clicking the a message to see, and randomly (I couldn't find any logic or system behind) it resulted an "invalid id:" error message. Then I was looking what request actually IE sends on the wire, and I found that IE sometimes added the payload to the POST query, however sometimes not. This happens at the settings page, while searching, and even when clicking on an email to view it.

However then I closed IE, started again, and did an ldap authentication (=not sso), and I couldn't reproduce the problem. It appears that IE + SSO login have this unfortunate side effect. Unfortunately I don't know why this is happening, no clue so far.

[Urs Frei]

Hmm, it's hard to believe on side effect cause it's exactly the same issue with all of IE's 8/9/10 and 11. Aso you can leave the IE SSO settings alone, just set no sso on config.php to get it work. But of course you know best where the code differences are with SSO. Otherwise think on functions where much grow with SSO and could bring the IE in trouble or it's a timing problem on IE or maybe it's the IE POST problem, about it you can read many posts on the Internet. I will also try to isolate the problem in more detail.

[Janos SUTO]

OK, I'll work on other things for a while, then I'll come back to this issue again with a fresh look, hopefully. And thanks for your efforts trying to nailing the issue down.

[Janos SUTO]

I've played a little more with it (using IE 11), and found that if you move 'piler.yourdomain.com' from "Local intranet" to "Trusted zones", then it improves the situation. Unfortunately this breaks SSO with IE. Still I'd like you to try it, and check if it helps the problem.

[Janos SUTO]

One more idea: try to upgrade jquery. Download it from http://code.jquery.com/jquery-2.1.1.min.js, then copy it over /var/www/piler/view/javascript/jquery.min.js, and see if it helps.

Update: perhaps it won't solve the problem.

[Urs Frei]

I had already tried without the "piler.yourdomain.com" in the IE "Local intranet" and of course i have do sign in with the netbiosname\username then, but the behavior is exactly the same as with on my site. The query-2.1.1.min.js was one of my first step to resolve the problem but did not help either. I was tried many others in the past too but without any better results, this was the reason to open a issue here. At the moment i have no more ideas too.

[Urs Frei]

I took a different path "network sniffing" between client and piler server and came across with other possible reasons. The INVALID ID: happens if the IE sends a POST /messages.php HTTP/1.1 , NTLMSSP_NEGOTIATE and the server response with data: text/html invalid id. The IE development team write about: Possible an unclosed <form> block or you are using <button> tags and not using return false; for the return value on the onclick event handler. On the other problem "searchfilter not working on first push off search button" the IE sends a GET //tmp/i.1764 (the number behind i. changes) and the server response with HTTP/1.1 403 Forbidden.

[Janos SUTO]

Can you try the following? Use the 'mobile' theme, and check again, please.

[Urs Frei]

It's the same behavior (INVALID ID and i have to push the search button twice). In the sniffer log i can't see any GET //tmp/i.xxxx now but this isn't the problem anyway. Today i was sniffing on one screen and was working with the IE on the other screen, so i could see what's happen at the same time. I saw it quite clear if any of our problem occurs the IE sends to the normal POST command an additional NTLMSSP_NEGOTIATE command without username and password. The IE makes it only if previously a NTLMSSP_AUTH has taken place.

[Janos SUTO]

I've checked for any unclosed form tags, and added "return false;" to all possible onclick sections. Still no cigar. I think the key is to fix IE / Outlook (I tested with Outlook 2013), so they wouldn't want to send authorization data other than /sso.php. As you figured it out, and I also ran ngrep to see what's going on. Outlook thinks randomly (1:10-20) that it needs to send ntlm auth data to apache. Too bad, it fails to send the POST data in these cases.

Anyway it's time to ask some windows experts for help. I posted the issue on a Hungarian forum, I hope someone may provide some clues. Until then unfortunately we have to consider SSO with IE or Outlook unsupported. I'll alert you as soon as there's any news about the matter.

[Janos SUTO]

One more thing to experiment with. http://stackoverflow.com/questions/4796305/why-does-internet-explorer-not-send-http-post-body-on-ajax-call-after-failure suggests that it's worth to try setting the keep-alive timeout value greater than 60 sec (=default windows timeout value).

KeepAlive On
KeepAliveTimeout 90000

After restarting apache I could click on 100 messages one after the other, and still no "invalid id:" error.

[Janos SUTO]

Perhaps the permanent solution is to leave the keepalive settings at apache alone, rather perform a registry hack + an IE options fix: http://support.microsoft.com/kb/251404 It needs some more testing, but so far it seems to work.

[Urs Frei]

I have played a little with the KeepAlive on Apache (MaxKeepAliveRequests to 0 and KeepAliveTimeout to 1000), it isn't the solution but makes the situation a little better. After SSO Login you can work normally with IE / Outlook until you make a longer break then the error happens once but after it you can work normally again. I was trying some registry settings for the IE (KeepAliveTimeout, ServerInfoTimeout, ReceiveTimout) either but without any better results. Maybe this information can help. Update: Apache KeepAliveTimeout to 100000 not 1000

[Urs Frei]

I was trying DisableNTLMPreAuth also but without any better result. I think it's for the IE6 only.

[Janos SUTO]

Besides the registry hack, I've also unchecked the "Enable Integrated Windows Authentication" option, then restarted the computer, and it apparently solved the problem, at least testing with Outlook 2013 + SSO.

[Urs Frei]

Hmm, on my site still the same, after SSO Login i can work normally with IE / Outlook until i make a longer break then the error happens once but after it i can work normally again. I do a search selection and leave the IE alone for 1 or 2 minutes if i open on of the Email then i get the Invalid ID but for one time only. Did you testing with fixed onclick sections on your site perhaps?

[Janos SUTO]

How long is that break? I tested with both the onclick fixed, and the original versions. It works both cases, and I don't think it's about the presence of "return false;" statements. I'm not sure if a user can or is willing to mess with the registry, but perhaps a workaround is simpler, ie. create a short text file, and retrieve it periodically before the keep-alive value expires. I'll revert the registry, and restore the options value I fixed, and try this approach as well tomorrow. I'll let you know the results.

[Urs Frei]

I hope you have understood me correctly, I take a break from work on IE for 1 - 2 minutes, after it the error happens just one time. Anyway, the workaround sounds good!

[Janos SUTO]

Hmm, I've played with it, and I think it's even better if you simply download a small text file (eg. ok.txt) synchronously, before I actually start the real ajax call. For the rest of the world of sane browsers this extra function will be empty.

[Janos SUTO]

OK, please download the latest master branch, and make sure you set the following in config-site.php:

$config['ENABLE_SSO_LOGIN'] = 1;

It activates a sort of keep alive feature for broken browsers (Explorer, Outlook).

[Urs Frei]

Thank you, it looks very good with the IE now. Outlook too but with a little side effect on the following scenario: go to the mailpiler folder in outlook, press search, go to an other oulook folder e.g. Inbox, go back to the mailpiler folder, press search again, open a mail from the list and at this moment the entire search result is cleared and you have to repeat the search.