Spam not being recognised second time around

[Roshan Jonah]

I have been moving 100s of spam emails from inbox to junk in Roundcube but the rspamd never seems to learn from them. I get almost exact same messages again in a couple of days. I have been doing this for well over a year now.

I assumed v2 has improved things but it doesn't seem to fix this particular issue. How do I add it to a cron job or something so it recognizes and stops these spam coming into inbox?

I am using the latest version 2.0.7 PRO. I also attached some of the sources of these emails for further investigation as to why they are not being recognized as spam.

[SH]

I am experiencing myself that autolearn is not working as expected. I will try to test it and tune asap

But your emails are strange. It seems you are receiving all this from google mailservers? It is some kind of redirect?

[Roshan Jonah]

Glad to know it's not just me experiencing this. It seems like they are from Google but I just saw that it is from Microsoft Hotmail servers too. Definitely not redirects. You can check the source files attached.

[SH]

Problem is, that very important part of antispam techniques is source reputation and sometimes it can adjust very negative rating to acceptable just because it has trustful source. So I am not entirely convinced that any content filter will work.

I've noticed that these emails are from google groups. Are you subscribed to some groups?

[Roshan Jonah]

It is actually sending to all gmail, icloud, and other such addresses but they are not from google groups. For example, this is not part of google group email. Our email address is not subscribed to any of the groups in Google. Content-filtering is a MUST if we were to win the SPAM war.

Return-Path: [e-mail] Delivered-To: [e-mail] Received: from mailsrv.**.biz ([127.0.0.1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by mailsrv.*.biz (Dovecot) with LMTP id VnHFHz76DFupXAAATNctgg for [e-mail]; Tue, 29 May 2018 06:59:46 +0000 Received: (mailsrv..biz); Tue, 29 May 2018 06:59:46 +0000 Authentication-Results: mailsrv..biz; iprev=pass; spf=none smtp.mailfrom=findyourmatch.date Received-SPF: None (mailsrv..biz: domain of findyourmatch.date does not designate 209.85.161.191 as permitted sender) receiver=mailsrv..biz; identity=mailfrom; client-ip=209.85.161.191; helo=mail-yw0-f191.google.com; envelope-from=[e-mail] Received-SPF: None (mailsrv..biz: domain of mail-yw0-f191.google.com does not designate 209.85.161.191 as permitted sender) receiver=mailsrv..biz; identity=helo; client-ip=209.85.161.191; helo=mail-yw0-f191.google.com; envelope-from=[e-mail] Received: from mail-yw0-f191.google.com (mail-yw0-f191.google.com [209.85.161.191]) by mailsrv.***.biz (Haraka) with ESMTPS id C8CB2BAB-0E3D-41CE-BEF4-E0FAE69236C4.1 envelope-from [e-mail] (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 verify=FAIL); Tue, 29 May 2018 06:59:37 +0000 Received: by mail-yw0-f191.google.com with SMTP id k186-v6so9378761ywc.18 for [e-mail]; Mon, 28 May 2018 23:59:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=findyourmatch-date.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:message-id:subject:mime-version; bh=KZWVfnVsL9RXG9xEkQX5nufvmoRZr0N1YAHjInPoY7s=; b=zSBfqSxBN/SdrlsfrOcKQsfW8uMm9U8W7l7Iw8xQlOYT40cbY1who660J0SZM/OarT DN775EnNDSt9xoupZ967SuF9HFKDij37fl5t4sxvZ/j0/TLtzHQj/5hZm13IL7WgNn7v dRUkOTBSP10tGYMljEW0S6VfovrDaWDJ9Kmr9S2q8JO0zam8vcz3NZ6UArUBFztW8gY/ 2dXLc2i5ynt1b7ysvppK2qYi0tUIrg+8IJ+jCZRWB2KSrat0aYQLnzz0Wxy54FHphpTD wdQHexyz4mpP54rwSvePSpA1vlRiTKaPX1tKt4vtKo2s6gMgHWhEFhlCnWj5oyz6PTtN bBug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-statefrom:to:cc:message-id:subject:mime-version; bh=KZWVfnVsL9RXG9xEkQX5nufvmoRZr0N1YAHjInPoY7s=; b=pINV4XgF/Z0H9GwK+eVk+27juMr4oWFgoEg8Jpfsd9kVoTMD2ZvJHOaixcH/B08+2/ BR6seEKvCyYOvBIP+h8rHKc4sEOHVRaKVAygRuQeZUORbAEB7TK8vTLeL3v5hJpcbsT2 KfBijpynUqElIO6Tco/GpnnOUZlgH5Z5odlyVx9G3pSZrsnR8qQ3HNEi+/sZR/QrlwqD pcnbbkorzMCcMUZQG76aEVQJssq5O/T0VOt6RW3bIR3kbIGdkbxCk8604KKENKJKNMNq 43+X6BHSsO20U+gLzhDzk0Z4JSSzDtDeCzgxgslwvoHKjFnuUHjWgjEPuUgZ8JMQFkhy O0Gg== X-Gm-Message-State: ALKqPwdsZlO22NpV11AxYd63VvRY6Vu9DUuKfKiZk1hlHdzmos/wpgVh TNRWQG29+n96FetkvdwmYgQa2HxPqKr0 X-Google-Smtp-Source: AB8JxZoFi72qJWOPgQZ4b/7wijFBfqn29RuTETMFCkKmmyLIAyoGMw4A4M58bvNmmbnMw78d6O6D4t9KIA== X-Received: by 2002:a25:cf13:: with SMTP id f19-v6mr530886ybg.5.1527577177058; Mon, 28 May 2018 23:59:37 -0700 (PDT) X-Google-Already-Archived: Yes X-Google-Already-Archived-Group-Id: 6cc45edb72 X-Google-Doc-Id: 68742f888107b X-Google-Thread-Id: 473ea20f036b2f73 X-Google-Message-Url: http://groups.google.com/a/findyourmatch.date/group/andrea-adams/msg/68742f888107b X-Google-Thread-Url: http://groups.google.com/a/findyourmatch.date/group/andrea-adams/t/473ea20f036b2f73 X-Google-Web-Client: true Date: Mon, 28 May 2018 23:59:36 -0700 (PDT) From: Andrea Adams [e-mail] To: Andrea Adams [e-mail] Cc: [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail], [e-mail] Message-Id: [e-mail] Subject: hello baby MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_36311_1178120486.1527577176694" X-Google-Token: ENj0s9gFdy0k9fyZJkg0 X-Google-IP: 202.134.9.137 X-Haraka-ASN: 15169 209.85.128.0/17 X-Haraka-Karma: score: 3, good: 8, bad: 1, connections: 11, history: 7, awards: 089,131,183,182, asn_score: -68, asn_connections: 2522, asn_good: 889, asn_bad: 957, fail:asn:history

[SH]

I agree - but still spam filtering is combination of various methods and source reputation can overwieght bad mail content. Anyway I will not close this issue until RSPAMD spam learning will be tested.

In meantime you can move this type of emails to Junk folder automaticaly by sieve filter which detects "X-Google-Message-Url" header or something like that...

require ["fileinto"];
if exists "X-Google-Message-Url"
{
      fileinto "Junk";
}

[Scott MacDonald]

Roshan.. are you forwarding your own emails to a gmail account? Or do you pull your email from your poste.io mailserver ?

[Roshan Jonah]

All the email accounts are coming straight from the poste.io mail server. None of them are forwarded to any of our Gmail accounts. So no forwarding is happening Scott.

[SH]

Strange thing is that your email is missing RSPAMD headers. Can you please paste here whole delivery log of one of this emails or send it to me to [e-mail]?

[Roshan Jonah]

Just did it.

[SH]

It seems that your rspamd is not working ok, can you please send me log/rspamd/rspamd.log also?

[SH]

ref #347 disable rspamd graylisting entirely

→ <<cset 1b45073b6bea>>

[SH]

@jonahnz can you please test latest release? I don't think it will solve the problem but there will be headers for debug inside mail so we will see better what is going on

[Roshan Jonah]

I just upgraded to the latest version. Here is the new header. I will post some more later on as they come through.

Return-Path: <[e-mail]>
Delivered-To: [e-mail]
Received: from mailsrv.inbox4.biz ([127.0.0.1])
    (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits))
    by mailsrv.inbox4.biz (Dovecot) with LMTP id kZaVNhrzF1tHZwAAMH1+8w
    for <[e-mail]>; Wed, 06 Jun 2018 14:43:38 +0000
Received: (mailsrv.inbox4.biz); Wed, 06 Jun 2018 14:43:38 +0000
Authentication-Results: mailsrv.inbox4.biz; iprev=pass; spf=pass smtp.mailfrom=trix.bounces.google.com
Received-SPF: Pass (mailsrv.inbox4.biz: domain of trix.bounces.google.com designates 209.85.214.77 as permitted sender) receiver=mailsrv.inbox4.biz; identity=mailfrom; client-ip=209.85.214.77; helo=mail-it0-f77.google.com; envelope-from=<[e-mail]>
Received-SPF: None (mailsrv.inbox4.biz: domain of mail-it0-f77.google.com does not designate 209.85.214.77 as permitted sender) receiver=mailsrv.inbox4.biz; identity=helo; client-ip=209.85.214.77; helo=mail-it0-f77.google.com; envelope-from=<[e-mail]>
Received: from mail-it0-f77.google.com (mail-it0-f77.google.com [209.85.214.77])
    by mailsrv.inbox4.biz (Haraka) with ESMTPS id B15061DC-B6A6-430A-9F4B-04D3B99B8EDF.1
    envelope-from <[e-mail]>
    (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256 verify=FAIL);
    Wed, 06 Jun 2018 14:43:29 +0000
Received: by mail-it0-f77.google.com with SMTP id r7-v6so5235141ith.5
    for <[e-mail]>; Wed, 06 Jun 2018 07:43:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
    d=daynightfun-science.20150623.gappssmtp.com; s=20150623;
    h=mime-version:reply-to:message-id:date:subject:from:to;
    bh=SXFj6igjuOfbJYzujciIVl3xAcUY+9WHKtZiANyuFo8=;
    b=VdFP/gPKppThUotBFKIoHH723b3iBDM4ojJysfeq7IKuCjDOm+/QFRQF1deJSteAow
    MW2KL4Uy+6s0ceQuinxhXbNitdT+ewZpUBtg7sATG4+Fnob35rhg64HgJQR3dMhV6j36
    37Gk1+rV/cdDwW5JiuZ5FQ0V3hHrJzYPrpekiyIj3SF7W0MB3srEj8QzC7aWqCq/Teaj
    iCtRBoD/FFGJasJy+Bnk6vvQ8m7fn7fcRatNmWsOtW93LXZdryt8bgxSXvfSaQfbSPcz
    wPfAw59vGpCKQQ5OdNgRnhuo4zgKW2ynuH2if8uSiDMKv5Z4gPxCtUUvO9aKQx+vadhz
    03lw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
    d=1e100.net; s=20161025;
    h=x-gm-message-state:mime-version:reply-to:message-id:date:subject
    :from:to;
    bh=SXFj6igjuOfbJYzujciIVl3xAcUY+9WHKtZiANyuFo8=;
    b=aNGvIwK6QJdCA0cbieBZXN/r4nWjHed72VKdfOWm2w9TDVimrlgZJP0h7PEm9NOXom
    P+kojIhRFSCkJBVaWINz/bXvTclwCQN0Ce6btOY5wgf44NmUK2AL/IZfH/CwiWz3/rPR
    O4vK0nDEwawHAfdb7ueON7+v61s/j2C5pnmSStuewZ5dDSv0R0an4ae44DZL1Es9eam/
    ESwunCryomZQaOteZOy0tkqfN6VeKyOCgxdl8d41z/qcX4g+Jyt2w4dBbmF+Zn4WsnVP
    20XJOkEtp86heSSvvUHVHdHGjb+48sE8RhoX5+Zr5VoXwIBlmcIa5Lb2C6/yFspvz8B5
    U/FA==
X-Gm-Message-State: APt69E2bUuWa1kBUwcY3PWNJKq+W0utOxfVDHfi2RMgioRxg/i+mbsyd
    4kvn7HNrEiQ8TnVCIz5SRCwoiHK5QhYQ+1enyb6EtFg=
MIME-Version: 1.0
X-Received: by 2002:a24:554:: with SMTP id 81-v6mt1822818itl.52.1528296208631;
    Wed, 06 Jun 2018 07:43:28 -0700 (PDT)
Reply-To: [e-mail]
X-No-Auto-Attachment: 1
Message-ID: <[e-mail]>
Date: Wed, 06 Jun 2018 14:43:29 +0000
Subject: Sexy woman for Hot mature men
From: [e-mail]
To: [e-mail]
Content-Type: multipart/alternative; boundary="000000000000b37cbe056dfa3028"
X-Haraka-ASN: 15169 209.85.128.0/17
X-p0f-Result: os="Linux 2.2.x-3.x" link_type="generic tunnel or VPN" distance=23 total_conn=1
X-Rspamd-Bar: /
X-Rspamd-Report: BAYES_SPAM(0.000204) R_DKIM_ALLOW(-0.2) FORGED_SENDER(0.3) URI_COUNT_ODD(1) MIME_GOOD(-0.1) IP_SCORE(-0.468482) R_SPF_ALLOW(-0.2)
X-Rspamd-Score: 0.331722
X-Haraka-Karma: score: 1, good: 0, bad: 0, connections: 15, history: 0, awards: 089,131,183,181, asn_score: -268, asn_connections: 3501, asn_good: 1159, asn_bad: 1427, fail:asn:history

[Roshan Jonah]

Here's the good news. After the upgrade, most seem to be ending up in the Junk folder. I didn't see many emails at all this morning.

[SH]

Ok closing. So it was dubious soft reject in rspamd (same as in my case).

(There is BAYES_SPAM score in header which means it went through bayes filter. Also it means that database of learned emails exists othervise there would not be any bayes score at all)