Issue #2: Upcoming Versions, Usage Limits, Writing Custom Rules
by Warren Togami
Welcome to Spamassassin News Issue #2. We now have 61 subscribers.
Prospects for Spamassassin-3.3.2 and 3.4.0
==========================================
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6392
Kevin McGrail of the Spamassassin PMC mentioned that he is trying to get
spamassassin-3.3.2 released during February in order to fix this bug
with perl-5.12+.
Kevin and I hope to enable a spamassassin-3.4.0 release in August 2011.
The most difficult requirement for a major release is to expand the
quantity and variety of the nightly masscheck corpus. This is critical
in order to accurately test and assign scores to the rules.
http://ruleqa.spamassassin.org/20110122-r1062119-n
We are seriously in need of more nightly masscheck volunteers. Currently
there are only ~12 participants. Do you really want your filter scores
to be based upon the mail of only 12 people?
We are especially in need of varied ham of non-geek and non-English
users. Many of you mail sysadmins are in a position where you might be
able to find such volunteers. A powerful incentive for an organization
or company to participate is Spamassassin becomes safer and more
effective for their own variety of incoming mail.
https://fedorahosted.org/auto-mass-check/
Learn how to get started in the nightly masscheck.
Status of Spamassassin-3.2.x: Unsupported?
==========================================
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6538
This might be of interest to those who continue to use
spamassassin-3.2.x. Apparently there are dangerous rules that the
project has no intention of fixing in the sa-update channel. Instead
they recommend everyone upgrade to the latest version.
http://www.spamtips.org/2011/01/spamassassin-32x-is-unsupported.html
I wrote some comments on this matter.
http://www.spamtips.org/p/rpm-packages.html
RHEL and CentOS 5 users are strongly encouraged to upgrade to my
spamassassin-3.3.x RPM which has been vigorously tested on many
production servers. Please report bugs to me.
DNSBL Safety Report 1/23/2011
=============================
http://www.spamtips.org/2011/01/dnsbl-safety-report-1232011.html
The latest DNSBL Safety Report looks more closely at rule overlap
because it can be an important factor in deciding an appropriate score
for a rule. I've found that raw performance numbers published on other
DNSBL comparison sites can be misleading as to the safety and
effectiveness of a list. The report now cautions against the use of
many DNSBL's due to reasons of ham safety or overlap.
DNS Whitelist - Is your Outgoing MTA Listed?
============================================
http://www.spamtips.org/2011/01/dnswl-please-list-your-mail-server.html
This post describes several benefits to listing your legitimate outgoing
MTA in DNSWL.
Disable Two Network Rules
=========================
http://www.spamtips.org/2011/01/disable-rfc-ignorantorg-rules.html
http://www.spamtips.org/2011/01/disable-dnsfromahblrhsbl.html
Masschecks have identified two network rules that are almost completely
ineffective. You might as well disable them and do two fewer DNS
queries per mail scan.
Know the Usage Limits of Spamassassin Network Rules
===================================================
http://www.spamtips.org/2011/01/usage-limits-of-spamassassin-network.html
If you receive more than 100K/day mail it is vitally important that you
read this article. Various network test providers might cut you off and
cause significant problems to your mail server if you don't know their
usage rules. This article describes the various limits and suggests
certain providers that are well worth paying for if you are a large site.
Ultimate Setup Guide
====================
http://www.spamtips.org/p/ultimate-setup-guide.html
The guide has grown a few sections since our last publication. Please
let me know if you have anything to add. We now have some instructions
for Debian and Ubuntu users.
Real Life Example: Writing Custom Rules
=======================================
I am especially interested if you have any effective custom rules to
share. I'd love to put your custom rule into the nightly masscheck
where it can be tested on the worldwide corpora currently of 460K spam
and 250K ham. If your rule is good, I will post it to the SpamTips.org
blog and it might make it into the official sa-update channel.
http://www.spamtips.org/2010/12/cacheredir-rule-prevent-google-cache.html
For example let us look at the SpamTips.org CACHEREDIR rule. I noticed
a few weeks ago that some spammers were bypassing the very effective
URIBL rules by using unusual Google cache URL's. If you click these
URL's you are redirected to a spam site from Google's cache.
http://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/wtogami/2...
Given that spam used non-standard cache URL's I wrote a regex to detect
this pattern. Masschecks indicated 1-4% of spam was abusing this rule,
with while it triggered on none of our sample ham.
http://www.spamtips.org/2011/01/no-more-cacheredir.html
This pattern disappeared in spam about a week ago even though Google
cache redirectors still are abusable. I suspect spammers knew we are
detecting their old pattern and have changed their syntax in a way that
is not detected by our current regex. I haven't found it yet though.
If you notice certain patterns that frequently appear in spam but not
ham, you can write your own custom rules. Put them in your local.cf
with score 0.01, then run spamassassin --lint to be sure your syntax was
correct. Test it on your own mail for a while. If it seems good, send
it to me and I will test it in the nightly masscheck.
Any Questions/Suggestions?
==========================
Please feel free to e-mail me if you have any questions or suggestions.
Warren Togami
warren(a)togami.com
13 years, 3 months
SpamTips.org Blog, DNSBL's, RPM packages
by Warren Togami
Hi folks,
This is the first time I'm posting to the spamassassin-news
announce-only list. We currently have 35 subscribers. Woo hoo!
Maybe I should rename this "SpamAssassin Weekly News"? Except it wont
be weekly. It will be whenever I feel like writing it.
SpamTips.org Blog
=================
http://www.spamtips.org
I started this new blog. Pretty much the same idea as this list. I
will occasionally notify this list if I feel a particular spamtips.org
blog post is important or really good. The blog will have some lesser
importance posts that aren't worthwhile to broadcast here on the list. I
am conscious of not posting too often to a list because I know it can be
annoying if the news isn't important. So if you read RSS regularly you
can subscribe to that blog and you don't need to follow this list.
There were two good posts so far.
http://www.spamtips.org/2010/12/cacheredir-rule-prevent-google-cache.html
Simple but effective rule that catches ~1-4% of spam that often slips
past SpamAssassin with a low score. This rule will continue to be
effective until Google has stopped people from abusing their cache as a
URI redirector.
http://www.spamtips.org/2011/01/dnsbl-safety-report-122011.html
Analysis and commentary on DNSBL's: Hostkarma, Spam Eating Monkey,
Tiopan, MailSpike, NiX Spam and PSBL based upon actual statistics.
How to Help Improve SpamAssassin?
=================================
http://www.spamtips.org/2011/01/dnsbl-safety-report-122011.html
Please read the bottom part of this blog post. SpamAssassin upstream is
in serious need of additional participants in the nightly masscheck.
Without a good sample size and variety of mail, SpamAssassin cannot test
the safety and effectiveness of rules and they are unable to make new
releases.
http://ruleqa.spamassassin.org/
As SpamAssassin server administrators you are in a unique position of
being most capable of running the masscheck script every night in cron.
Masscheck's log file is uploaded and combined with the logs of other
participants in order to generate the RuleQA statistics you see here.
We are especially in need of non-English or non-technical mail.
RPM Packages
============
http://www.spamtips.org/p/rpm-packages.html
Here I will post the RPM packages of the latest spamassassin and related
anti-spam stack. It certainly isn't "official" but I personally use
these packages in production, so I guess it is good enough. If you use
my packages you may report bugs to me.
If you are running RHEL-5 you really want to upgrade to my spamassassin
RPM. It sucks a lot less than the spamasssasin-3.2.x in RHEL-5.
RHEL-6's spamassassin-3.3.1 is good enough for now. I'll let folks know
when I have something clearly better for RHEL-6.
Ultimate Setup Guide
====================
http://www.spamtips.org/p/ultimate-setup-guide.html
I haven't worked on this page much yet, but I will eventually put *ALL*
of the current SpamAssassin configuration advice here. This way you
wont need to read through hundreds of blog posts to understand the
current (and non-obsolete) configuration advice.
Check out the part about NJABL and rfc-ignorant.org, where you can
improve your network query efficiency a small bit with a config tweak.
Any Questions/Suggestions?
==========================
Please feel free to e-mail me if you have any questions or suggestions.
I am especially interested if you have any effective custom rules to
share. I'd love to put your custom rule into the nightly masscheck
where it can be tested on the worldwide corpora currently of 460K spam
and 250K ham. If your rule is good, I will post it to the SpamTips.org
blog.
Thanks!
Warren Togami Jr.
warren(a)togami.com
13 years, 3 months