Spam

From HerzbubeWiki
Jump to: navigation, search

Overview

This page discusses my experiences with the Spam problem.


Mail server configuration

Details about my mail server configuration can be found on these pages:


Related/interesting pages


Spam statistics

I did not yet get around to writing a script that collects data from my mailbox so that I can automatically generate spam statistics. I have therefore decided that whenever I clean out my spam folder, I will note down details about the state of affairs in the following table. Each line contains information about the period of time that has passed since the date of the previous entry.


Date Days elapsed Spam messages received Messages/day Correctly classified by SpamAssassin False negatives False positives Notes
Manually trained DNS blacklist warning
04.06.2008 n/a 46000 n/a n/a n/a n/a - -
12.08.2008 70 47365 677 46354 (97.9%) 1011 (2.1%) n/a - -
30.09.2008 49 39099 798 38222 (97.8%) 877 (2.2%) n/a - -
07.12.2008 68 45088 663 44177 (98.0%) 911 (2.0%) n/a - For a couple of weeks, the daily amount of spam had decreased significantly. I guess I have been experiencing the direct result of web hoster McColo being taken off the net. Unfortunately, the rate has been getting back to "normal" (see this story about the spammers' backup plan).
12.02.2009 67 45624 681 44733 (98.0%) 891 (2.0%) n/a - -
23.04.2009 70 47366 677 46254 (97.7%) 1112 (2.3%) n/a - Didn't train for the last 40 days (while on freighter travel)
10.06.2009 48 49481 1031 47965 (96.9%) 1516 (3.1%) n/a - I have trained often, even though I have been travelling, but still the rating of unrecognized spam has gone up and is, in fact, worse than during the previous period where I did no training at all. Two possible reasons for this are:
  1. There was an overall surge of spam, the message/day ratio has gone up by 52%!!!
  2. For some time sender addresses from the herzbube.ch domain were abused, resulting in an increased number of collateral spam ("undeliverable message" etc.) that got through the filter.

On a side note: I pruned the auto-whitelist database, which had grown to massive size over the years, but this should not have had an influence on the number of unrecognized spam.

30.07.2009 50 58462 1169 55520 (95.0%) 2942 (5.0%) n/a - The basic "spam message per day" ratio has increased again, but what is even worse: More spam than ever has passed by the filter, the average is now 60 spam messages per day in my inbox :-( Will this terror never end?
07.09.2009 39 45899 1177 44320 (96.6%) 1579 (3.4%) n/a 1 The picture remains unchanged, but today I have finally, reluctantly, implemented greylisting. It will be interesting to see how greylisting affects this whole spam affair. An interesting number on the side: Of all the spam messages I received, 3796 (8.3%) had a recipient that contained "iana.pen". This pretty much says everything about address harvesting... Another side note: Today I added the "False positives" column because in the previous period I had one of these. In earlier periods the column says "-" because I have no reliable numbers. However, if I recall correctly, I have had only 2-3 false positives in all the time since I am using SpamAssassin (between 5 and 6 years).
30.10.2009 53 5223 99 4872 (93.3%) 351 (6.7%) n/a - After almost 2 months I conclude that greylisting is the most effective anti-spam measure that I have ever seen: Implementing it reduced the message/day rating by an impressive 92%. Of all the spam that still came through, 4568 (87.5%) messages were delivered via the backup MX (virusscan.solnet.ch) which unfortunately does not implement greylisting. I have now temporarily removed the backup MX entry from my DNS configuration (and reset the greylist daemon's whitelist) - it will be very interesting to see the results of this latest experiment.

Update on the iana.pen statistics: 358 (6.9%) messages had the string "iana.pen" in the To: header.

18.12.2009 49 992 20 699 (70.5%) 293 (29.5%) n/a - After another 7 weeks of running entirely on a diet of greylisting (i.e. the backup MX was turned off all the time), the numbers look even better: The message/day rating went down by another hefty 80%, if compared with the ratio of the pre-greylisting era the improvement is now over 98%!!! An interesting observation is that the effectiveness of greylisting has lowered SpamAssassin's recognition percentage. It appears that spammers who are capable of circumventing greylisting are also better with crafting "quality" spam that can fool SpamAssassin. My new goal therefore is to raise SA's recognition rate to >=95%.

iana.pen statistics: 68 (6.9%) messages had the string "iana.pen" in the To: header.

10.03.2010 82 2441 30 2078 (85.1%) 363 (14.9%) n/a - In the almost 3 months since the last count, SA's recognition rate has significantly increased, probably due to the longer sampling period and therefore a better average. Although I figure I could improve the rate still further by tweaking SA parameters more aggressively, I do not want to risk any false positives. At present, I therefore let the matter stand as it is.

iana.pen statistics: 143 (5.9%) messages had the string "iana.pen" in the To: header.

14.12.2010 280 10133 36 9554 (94.3%) 579 (5.7%) n/a 4 With 9 months this has been the longest sampling period since I started this statistics page! I'm glad to see that SA's recognition rate has further improved without effort on my side - isn't this what computers are supposed to do: Lifting the burden of work from man's shoulders? :-)

iana.pen statistics: 1563 (15.4%) messages had the string "iana.pen" in the To: header.

05.01.2012 387 7393 19 6715 (90.8%) 678 (9.2%) n/a - Slightly more than a year has passed since the last sample. In this time the message/day spam rate has dropped to an all-time low. It is unclear whether the reason for this is a world-wide decrease in spam mails, or a decrease in the "quality" of spam mails, i.e. fewer spam mails make it past the greylisting wall. My gut feeling is that it is the latter. Although less overall spam is good, SpamAssassin's recognition rate has dropped by almost 4%. This makes for 1.75 spam mails per day in my inbox, which is still less than the 2.06/day average of the last sampling period (due to the low overall spam rate).

iana.pen statistics: 319 (4.3%) messages had the string "iana.pen" in the To: header.

07.06.2013 519 7308 14 6576 (90.0%) 732 (10.0%) n/a - 17 months after the last sample, I'm pleased to see that the message/day spam rate has dropped again. The average number of spam mails that have made it into my inbox is now at 1.4 messages/day. The SpamAssassin recognition rate is still about the same, but I guess it's hard to have a better rate without resorting to black lists.

iana.pen statistics: 481 (6.6%) messages had the string "iana.pen" in the To: header.

16.01.2015 588 6252 11 5301 (84.8%) 951 (15.2%) n/a - Another long sampling period (19 months) and the message/day spam rate has dropped for the third time in a row. So far so good, but unfortunately at the same time SpamAssassin's recognition rate has dropped for the third time in a row. This time the drop was so marked that, although the total number of spam messages is lower, the average number of spam mails that have made it into my inbox has increased to 1.6 messages/day - the first increase since I switched to greylisting. This turnaround is important - and a little scary - because for the first time in almost 5 years spammers have actually become better in getting their junk into my inbox. I sincerly hope this trend will not continue.

iana.pen statistics: 509 (8.1%) messages had the string "iana.pen" in the To: header.

21.05.2016 491 5859 12 n/a n/a n/a - Roughly 16 months have passed since the last sample. For the first time since December 2010 the message/day spam rate has slightly increased. Unfortunately I can't say anything about the quality of spam because in the catastrophic server failure that happened on May 21 I have lost all information in this regard.

iana.pen statistics: 433 (7.4%) messages had the string "iana.pen" in the To: header.

26.07.2016 66 306 5 241 (78.8%) 62 (21.2%) n/a - This date marks resumption of regular mail service on pelargir.herzbube.ch, from now on with active DNS blacklist checks thrown into the mix of anti-spam measures - it will be interesting to see what effects this has on my spam statistics.

66 days have elapsed since the catastrophic server outage on May 21 this year. The statistics of this period certainly cannot be used for comparisons, because those 66 days are a mixture of

  1. No emails received at all for a few days after the server outage
  2. Emails received by a mail hoster (switchplus.ch), an emergency replacement of my own email service; and
  3. Emails received by the re-established but still experimentally configured email service on pelargir.herzbube.ch.

iana.pen statistics: 39 (12.7%) messages had the string "iana.pen" in the To: header.

25.05.2017 303 7309 24 6604 (90.4%) 510 (7.0%) 195 (2.7%) - 10 months since the last sample. The overall spam rate has increased significantly, since 2012 there hasn't been a higher spam message/day ratio. One possible explanation for this is that I have somewhat relaxed my greylisting configuration: I am now running with a greylisting delay of only 1 minute instead of the previous 10 minutes, and I also have whitelisted amazonses.com. The good news is that the rate of spam that made it into my inbox has not substantially increased: I had to manually train 510 messages, which is 1.7 messages/day.

iana.pen statistics: 605 (8.5%) messages had the string "iana.pen" in the To: header.

And now for the real news: How did DNS blacklists influence my spam "experience? First, a short summary of my DNS blacklist policy: If a sending host is on at least two of the four DNS blacklists that I am using, I am outright rejecting any traffic from that host. If a sending host is on only one of the blacklists, I accept its traffic, apply the usual heuristics to the messages it sends, then place any messages that are still classified as ham into a special "DNS blacklist warning" folder. The statistics here look like this:

  • 195 messages were actually spam, i.e. false negatives. 150 were blacklisted by Barracuda, 20 by SpamCop and 25 by Spamhaus. I'm using this little command to count: grep X-DNSbl-Warning * | sed -e 's/.*blacklisted by //' | sort | uniq -c
  • 14 messages were ham, i.e. correctly classified. 1 message was from the Mantis bugtracker (mantisbt.org, blacklisted by SpamCop), two were Git commit messages from the RCMCardDAV mailing list (blacklisted by Barracuda), another one was a library newsletter (blacklisted by SpamCop), and the remaining 10 were notifications from Facebook (all blacklisted by SpamCop).

So to calculate the overall spam recognition rate we have to add those 195 messages to the 510 that I had to manually train. The overall rate therefore is 9.7%, which is not too bad in itself, and also those 195 messages didn't make it into my inbox, which is even better. I am a little concerned about the 14 ham messages which also didn't end up in my inbox, but none of them was really important, so my concern is not too great at the moment.