Delay sending change notification mail

Issue #646 resolved
Anonymous created an issue

When a change is committed to any of our Subversion repositories in SCM Manager it takes around 45 minutes before change notification emails are sent out. When received the correct commit time is shown in the content of the mail.

Here are some example log entries filtered by 'DefaultMailService' (note the time gaps):

2014-11-27 14:21:57.820 [Hook-1] TRACE sonia.scm.mail.spi.DefaultMailService - no from recipient found setting default one: scmmanager.rsdevbuild@bl.uk
2014-11-27 14:44:23.402 [Hook-1] DEBUG sonia.scm.mail.spi.DefaultMailService - send email to DLRemoteServicesJavaDevelopment@bl.uk from scmmanager.rsdevbuild@bl.uk
2014-11-27 15:06:22.405 [Hook-1] TRACE sonia.scm.mail.spi.DefaultMailService - no from recipient found setting default one: scmmanager.rsdevbuild@bl.uk
2014-11-27 15:06:22.405 [Hook-1] DEBUG sonia.scm.mail.spi.DefaultMailService - send email to christian.peacock@bl.uk from scmmanager.rsdevbuild@bl.uk

These all relate to a Subversion check-in that was made at 14:21:57. Neither of the emails were received until 15:06.

This is causing us problems as the delay can mean conflicts arise when other developers are unaware of changes made in a timely manner.

Comments (12)

  1. Chris Peacock

    I don't know but can find out if necessary.

    However, I can send an email instantly from the same server to the same recipients using Telnet to the SMTP server. I've also tried two different SMTP servers.

  2. Chris Peacock

    Hi Sebastian,

    Is there any chance you could get to look at this?

    A possibly related problem is that SCM Manager is regularly taking up almost 100% CPU time on our server and requiring killing and restarting. This seems to happen more after several branching and committing operations have taken place and it's becoming something of a problem. When the process is killed the notification emails are never received; the rest of the time they take at least 45-50 minutes.

    In answer to your previous question there is no greylisting. In any case, we have Jenkins running on the same server and sending emails via the same SMTP host which arrive instantly.

    Thanks.

  3. Chris Peacock

    I've investigated this and it seems that the problem most likely lies in the class org.codemonkey.simplejavamail.EmailValidationUtil.

    Our SCM Manager notification email address is 37 characters long. I've written a test script to call the above class, and when called with email addresses over about 31 characters long, the time taken for the method 'isValid' to complete appears to grow exponentially for each increase in length by a character. When called in my script, the method does return eventually, so perhaps SCM Manager is timing out waiting for it.

    I imagine the cause is in the regular expression generated within the email address validation, which in the case of my test script is:

    ((([ \t][a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+[ \t] )|(\"([ \t]([\x01-\x08\x0B\x0C\x0E-\x1F\x7F\x21\x23-\x5B\x5D-\x7E]|(\[\x01-\x0 9\x0B\x0C\x0E-\x7F])))[ \t]\"))+)?[ \t]<(([ \t]([a-zA-Z0-9!#\$\%\&\'*+- \/\=\?\^_`{\|}\~]+(.[a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+))[ \t])|(\"([ \t]([\x01-\x08\x0B\x0C\x0E-\x1F\x7F\x21\x23-\x5B\x5D-\x7E]|(\[\x01 -\x09\x0B\x0C\x0E-\x7F])))[ \t]\"))@([ \t]([a-zA-Z0-9!#\$\%\&\'*+-\/\=\? \^_`{\|}\~]+(.[a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+))[ \t]|\ [([ \t][\x01-\x08\x0B\x0C\x0E-\x1F\x7F\x21-\x5A\x5E-\x7E]|(\[\x01-\x09\x0B\x0C \x0E-\x7F])+)[ \t]])>|(([ \t]([a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|} \~]+(.[a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+))[ \t])|(\"([ \t]([ \x01-\x08\x0B\x0C\x0E-\x1F\x7F\x21\x23-\x5B\x5D-\x7E]|(\[\x01-\x09\x0B\x0C\x0E- \x7F])))[ \t]\"))@([ \t]([a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+(\ .[a-zA-Z0-9!#\$\%\&\'*+-\/\=\?\^_`{\|}\~]+))[ \t]|[([ \t][\x01-\x08 \x0B\x0C\x0E-\x1F\x7F\x21-\x5A\x5E-\x7E]|(\[\x01-\x09\x0B\x0C\x0E-\x7F])+)[ \t ]])

    This ties in with the behaviour of SCM Manager, where I've managed to reproduce the problem using the email Test Configuration button. When testing using a recipient address up to 30 characters the test works fine, but above 30 I receive the message "Sending test email failed."; above 32 characters the message is still shown and the Commons Daemon Service Runner process on the server sticks at around 50% CPU usage and never drops. If the test above 32 characters is run a second time, CPU usage climbs to nearly 100%. It is this behaviour that is causing serious problems for us, as SCM Manager resides on our build server.

    Is there a possibility of SCM Manager switching to using a different email library?

  4. Vadim Thesis

    Hi. It looks like I encountered this issue with 1.54. My notification email is like '4charsprefix_my-repo-pretty-long-name@vcs.local', 43 chars total. Upon commit Java raises CPU load up to 100% (I waited for 1+ hour before restarted tomcat). Notifications still working fine for short email addresses.

  5. Log in to comment