Ruthie Toce
May 17, 2019
Why Is Spam Filtering Not 100% Accurate?
Do I need spam filters?
Spam filtering for small business is an important part of basic cybersecurity best practices. Imagine if every morning you went to check your inbox and had to rifle through hundreds of emails – most of which were junk… ouch!
Quick aside: I actually had a client tell me years ago (pre-Velo of course!) they had the “human spam filter”, an early-riser secretary that would come in every morning at 6am and delete junk emails and then forward legitimate emails out to the correct user! Wow… the epitome of efficiencies (read: sarcasm)!
Moving on… think about all the legitimate messages you’d miss while deleting the trash with this manual approach! We seem to take them for granted but day in and day out spam filters keep us organized, protected, and maybe even a little saner than we would be without them.
First... What is spam?
Spam, quite simply, is unwanted, unsolicited, repetitive, and mostly unavoidable email from unknown senders. Spam has evolved over the years into more than just an annoyance. Today, spam more often widely describes all email-based threats such as phishing and spoofing. These threats are typically trying to get information out of you (banking info, usernames and passwords, etc.) so that you, your business, and your identity can be compromised.
What do spam filters check for?
With each spam message that lands in your inbox, the more susceptible you are to cyberthreats. Sanity aside, every business owner wants to protect both their employees and their data as best as they can. Spam filtering for small business can help. Spam filters can and should be configured to best fit your business needs, however, they generally look at three main areas:
- Header Data
- Blacklists
- Email Content
What is email header data?
Every email contains header data the average end user will never see. This data contains information such as server IP addresses to show where the email has been, where it came from, and where it is going. It contains timestamps, and other information about the formatting of the message, as well as the addresses of the email senders and recipients. Spam filters check this data for indications that the message could be spam, such as the time stamp not making sense for when the email was actually sent, if any of the IP addresses associated with it are blacklisted, or if the sender or recipient addresses are invalid.
What is a blacklist?
Blacklists are lists of known spammers or spamming sources. These known spammers can be automatically blocked from being delivered to your inbox by spam filters. Who makes these lists? Typically, Internet Service Providers (ISPs), server admins, and email providers compile these lists from data all over the world. As spam filters detect spam based on content analysis, security alerts, or maybe a flood of emails from a single particular source, the source of these spam emails is tracked and placed on an email blacklist. Once a source is on a blacklist, the spam filters that reference this blacklist will typically start to block any emails coming from that blacklisted source.
Email Content Analysis
Another main area spam filters check is the content within the email. Filtering by the content is where this otherwise simple filtering system gets tricky. Where some things are black and white – blacklisted and not blacklisted, valid address or invalid address – content is all rainbows and light prisms. Of course, there are a few stand out elements that are known spam indicators. Things such as blacklisted websites, executable file attachments (file types that can perform multiple functions and operations) or known spam keywords contained in the email are all dead giveaways of spam. These content guidelines, much like the entire IT industry, are always changing. This fluid motion of what is and isn’t considered spam is what makes it so hard for spam filters to filter the good and bad properly. For example, short codes used to be flagged as spam, but with the rise of social media, Twitter especially, short codes have become fairly regular visitors in the body of an email, making it hard to now say that all short codes indicate that the email is spam.
No hard and fast rules
So how, after everything spam filters check, are they still not 100% accurate? Well first off, we’re talking about technology. The only certainty about most technology is its uncertainty! On top of that, spam filters are working against both computers and the human brain. As the industry evolves and scales, so do cybercriminals. Every article you read about enhanced cybersecurity, they read, too. Spammers know what is flagged when filtering emails and know how to adapt to better trick the system and get past your spam filters.
As we now know, spam filters check a number of components within a single email before marking it as spam. Most spam filters score email messages and add up points for each thing that could indicate spam. If the email scores high enough, then the spam filter will quarantine or block that email and keep it out of your inbox. If only one part, such as the To: field is scored as potentially spam, but everything this else checks out, that email is most likely still making it to your inbox. For instance, if you email a colleague and forget to add in a subject line, your email will typically still be delivered. However, if an email has no subject, a sender name that doesn’t match its email address, and has a questionable attachment, chances are that email will be marked as spam and never make it to your inbox. If you would set a hard and fast rule that ALL emails without a subject line are blocked, you’d probably miss out on a number of emails you want to receive. Although these gray areas allow you to get the emails you want (yay!), it can also lead to the bad emails getting in (boo!). This is where configuring your spam filters to best suite your specific needs can work wonders for your business.
My emails are getting caught in spam, what can I do?
So maybe you’re on the sending side of emails more often than the receiving – working on email marketing campaigns and want to know how to get your email into people’s inboxes. For this, it’s still good to understand how spam filters work. You don’t want to spend weeks or months working on a project only to have your hard work get caught up in a spam folder. There are a few things you can do to help in these situations. Make sure your content is clean and concise, test any links included in the message, and keep visual branding consistent with your website and social media channels. Additionally, have your subscribers opt-in to receiving emails from you, and comply with required methods for allowing people to opt-out if they so desire. If your messages are still getting blocked, A/B testing should help figure out what the holdup is.
Spam filtering is not perfect, but could it be?
All of this is to say that yes, spam filters are necessary. They do a wonderful job at keeping spam out of your inbox and the cyberthreats associated with them at bay – whether they are perfect or not. For the most part, spam is free to send and easy to blast out in large volumes. The sad truth is that with just a .1% open rate, spam remains profitable enough for those in the biz to keep on trucking along. Just as we mentioned in our phishing blog, those few gullible people that fall into the spam trap make it all worthwhile.
Although spam filters are not 100% accurate now, does not mean they never will be. The more technology evolves and grows, so does our ability to apply those innovations to spam filtering. As more spam emails and data surrounding them is gathered, the more we can learn. Algorithms are being created from all the emails marked as spam and are used to help block potential spam messages. Even cloud computing can help tighten up spam filtering. The scale at which cloud providers can collect data, and their accessibility to a wider audience, make for stronger content filtering that more spam filtering companies can use.
Advanced Methodologies
Further, advanced email filtering methodologies are being more widely adopted and improving filtering results. Technologies such as DKIM (DomainKeys Identified Mail) affix a digital signature to emails coming from senders to validate the email is really from who it says it is from, while SPF (Sender Policy Framework) leverages DNS technology to try and validate email is coming from a valid source. Most recently, DMARC (Domain-based Message Authentication, Reporting and Conformance) is being adopted by many organizations to extend the effectiveness of DKIM and SPF, further thwarting cyber-attackers from spoofing email address and impersonating your CEO!
Spam filtering, even though it’s not 100% accurate all of the time, is a great way to protect your business from common cyber threats and it can only get better from here. If you have any questions about these filtering technologies we have discussed, or want to learn more about how your business might be able to improve its email filtering and security posture, feel free to reach out, Team Velo is always here to help!
If you liked this article, check out these related articles:
