How to not get caught in spam filters

Reliably sending email without getting caught in spam filters is a full-time job, for someone. Surely not for an end-user, but for every end-user email, there is an administrator somewhere who has to deal with daily occurrences of some user message not getting through because it got stuck in a spam filter on the other end.

At the enterprise level, this could easily be several people's full-time jobs. Spam filtering is constantly evolving. This is partly due to new spam filtering initiatives that require administrators to configure something new, such as SPF or DKIM. A few years ago, SPF didn't exist. Now, anyone who sends lots of email virtually has to implement it. It's also partly due to other administrators; sometimes you just have to get on the phone with the recipient's admin to figure out what's going wrong.

This guide is not for those enterprise admins. It's for the hapless developers pressed into Postfix config duty for a small start-up, or for the first time admin just getting into outbound mail. What follows is a quick and dirty guide to making sure 99% of your email is delivered.

Make sure you're not on a DNS blacklist (aka RBL: Reverse Blacklist)

By far the most frequently used type of spam filter is the DNS blacklist. There are hundreds of free services out there that keep records of IP addresses they think send a lot of spam. Virtually every spam filtering product on the market comes pre-configured to look at a few of these every time they get a new connection. It's fast due to extremely low over-head (DNS scales, baby), and relatively accurate.

You will need to know what IP you're sending from. You can check many blacklists at once via various different sites.

If you are on a blacklist, you might be wondering how to get off it, and how you got on in the first place. Unfortunately, there is no single answer. Each blacklist has its own criteria for who it lists, and has its own process for removal. Indeed, many lists don't allow removal at all. It's the wild-west out there. If you find yourself unable to be removed from a popular blacklist, you may have no choice but to buy another IP address. Just make sure it's clean first!

Some people think blacklists are the devil. If you have ever found yourself at the mercy of a popular, but totally non-responsive blacklist, you might agree. But in general, the problem is that some administrators outright block email that matches a single blacklist. If you're an inbound admin, don't do that! You want to weigh many factors, and multiple blacklists, before you decide to reject a message. Regardless, they are a reality of the modern Internet you need to just deal with.

Make sure you're not an open relay

If you want to STAY off blacklists, you at the very least need to make sure you're not an open relay. Basically, you should not accept and definitely not send out any mail that's not destined for a domain you actually own. Testing can be done via telnet, or via a web-based tool.

Reverse DNS (aka PTR records)

Another very common check is whether your IP address is named, or unnamed. The idea here is that dynamic IPs, such as those given to home users by their ISP, generally don't need to have names associated with them. A lot of spam these days comes from zombied home machines.

This is a simple DNS fix. You just need to create an PTR record for that IP address. You can check if your PTR is setup correctly with the following command.

dig -x MY_IP_ADDRESS

MX Records, postmaster, root & abuse

While the standards RFCs don't require you to receive mail just because you're sending mail, in reality many anti-spam systems are biased against message from a domain that does not also accept mail. You don't have to send and receive from the same server(s), but if you're sending mail from @example.com, it's a good idea to make sure some real human somewhere is getting any messages sent to postmaster@example.com, root@example.com and abuse@example.com.

Postmaster IS strictly required by the RFC. Root is a legacy version of postmaster. Abuse is a relatively new "standard" that many administrators would try first to resolve a spam issue.

Inbound email is a whole other subject. But the basic gist is that you need an MX record for example.com, and it needs to point to a server that can accept mail for example.com. If you don't have an existing inbound server, or don't want to run your own, many hosted alternatives exists.

You should explicitly test postmaster, root & abuse manually via your regular email client to make sure they actually work.

HELO, I'm your mail server

Mail servers communicate via a protocol called SMTP. It's actually a plain-text protocol, which you can easily emulate via telnet. The very first line of a SMTP handshake is the "HELO" command, where the sending server identifies itself. A typical example would be "HELO example.com", meaning, "Hi, I'm the mail server for example.com".

Many spammers set this to a bogus value, or try to use the recipient's host name or IP address, which is nonsensical. In any case, the correct thing to do is for you to set it to your domain.

How you set this will vary by mailserver. In Postfix, it's the myhostname parameter in /etc/postfix/main.cf. Checking it is easy; just send a message through the server, and look at the headers on the remote end. Your hostname will show up on the first "Received" header line:

Received: by example.com (Postfix, from userid 0)
 id A72979E4144; Thu, 18 Mar 2010 23:00:01 -0400 (EDT)

SPF/DKIM

SPF and DKIM are newer standards that are slowly gaining popularity. The basic idea is that your DNS records can encode a list of rules about what IP addresses are allowed to send mail for your domain. It's a whitelist, versus a blacklist. Typically, you can ignore these unless you're sending a large volume of mail.

Monitoring

That just about covers anti-anti-spam 101. As mentioned, this will likely be an ongoing effort, and you need to keep on top of how it's going. Ideally, there would be an administrator who would be alerted if emails are bouncing due to spam filters. For postfix, I would recommend pflogsumm.

apt-get install pflogsumm

sudo crontab -e

...
# every work-day at 11pm
00 23 * * mon-fri cat /var/log/mail.log |/usr/sbin/pflogsumm -d today |mail -s "daily mail log" postmaster@example.com