Scanner - A Content Scanner for Qmail (original) (raw)

Last Updated: Wed, 20 Nov 2013 21:45:18 +0000 GMT

Copyright 2011 Jason Haar. This software is distributed under the terms of the GNU General Public License. See for additional information.

Description

Qmail-Scanner is an add-on that enables a Qmail email server to scan gatewayed email for certain characteristics (i.e. a content scanner). It is typically used for its anti-virus and anti-spam protection functions, in which case it is used in conjunction with external scanners. It also enables a site (at a server/site level) to create "Policy blocks": i.e. react to email that contains specific strings in particular headers, or particular attachment filenames or types (e.g. *.VBS attachments).

Qmail-Scanner is integrated into the mail server at a lower level than some other Unix-based virus scanners, resulting in more thorough coverage. It is capable of scanning not only locally sent/received email, but also email that crosses the server in a relay capacity. Qmail-Scanner also leverages the wealth of meta-information provided by Qmail (such as client IP address, and whether or not the client is allowed to relay).

Features

• Supports almost all commercial (Unix) virus scanners as well as the ever-popular Open Source ClamAV scanner.
• Can call more than one virus scanner for each mail message
• Has its own internal scanner that can be used for Policy enforcement, or to quarantine viruses that your AV currently cannot detect
• The internal scanner can also be used to quarantine email based on attachment types, or email with certain email headers... Need to stop *.mp3 files or "**Subject:**ILOVEYOU" email getting onto and off your LAN - can do! :-)
• The internal scanner can trigger a "greylist" action instead of a quarantine. This is designed for emergency situations where your current AV and static Policy blocks are not appropriate. e.g. a new ZIP-based virus comes out with random filenames. Your AV cannot detect it, and you can't globally block ZIP files without hurting valid users. A "greylist" action will cause Qmail-Scanner to exit with a SMTP temporary failure instead of delivering the message. Valid emails will simply be requeued and can flow through later once your AV can detect the virus, and you decide to remove the greylist policy.
• Internal engine scans for poorly formatted messages that are known to be used by trojans/virii to infect clients. As such, this is independent of any virus scanner, and can successfully operate against future virii/trojans. Such messages are quarantined immediately. Known to block such major virii as Klez and Aliz, and as a side effect, stops a fair amount of spam too! Format checks include:
  • broken MIME continuation headers
  • use of comments within standard headers (e.g. "Content-T(xxxxxx)ype:" is *identical* to "Content-Type:" according to the RFCs - but some virii use this as it circumvents some anti-virus scanners). Valid use of this is never seen in the wild - so it's blocked
  • repeated occurrences of MIME headers makes Q-S rename the latter ones to nullify them
  • MIME boundaries over 250 chars are blocked
  • differing definitions of a particular attachment filename causes it to be blocked
  • double-defining the same MIME boundary is blocked
  • certain MIME types containing windows executable extensions are specifically blocked (e.g. an "audio/wav" of filename "wav.exe" could only be a virus)
  • broken headers within a MIME attachment are blocked
  • windows executable attachments that aren't marked as being of MIME type "application/....." are blocked (e.g. renaming notepade.exe to notepade.gif and sending it as a GIF attachment would be quarantined, as Qmail-Scanner would realise it's an executable pretending to be something else).
  • attachment filenames over 256 chars are blocked
  • some double-barreled filenames are blocked (e.g. file.gif.exe). It tries not to block common mistake variants
  • CLSID file extensions are blocked
  • Password-protected zip files can be blocked if you wish. This would stop any future viruses stuffed inside password-protected zip files from getting through, but of course would also stop any legitimate usage. Turned off by default, but perhaps useful to turn on during a new outbreak, and turned off again once an AV update occurs that can catch it.
• defaults to always running any AV you may have over messages first, then runs the internal scanner (Policy/perlscan) checks. This means if you block ".PIF" files due to them normally containing viruses, then any .PIF files that do contain a virus known to your AV system will be flagged as "viruses", and any that were missed (perhaps they were a Day-Zero virus) are then tagged as being blocked by "policy". This differentiation is then used by the alerting system. It defaults to not notifying the sender that a virus has been found, but can still notify them when it was a "policy" block.
• Quarantines emails it finds to contravene the above sub-systems. Viruses are quarantined into a maildir named "viruses/", policy-blocks into "policy/" and (potentially) high-rated SPAM into "spam/"
• Email that is quarantined can be configured to trigger one of two different forms of notification:
  1. defaults to generating a localizable (i.e. language support) email alert notifying the sender and/or recipient why their message was quarantined.
  2. a SMTP reject error message (5XX) if desired. This is disabled by default. Needs the Qmail Custom Error patch if you want nicer SMTP error descriptions
• Can integrate with SpamAssassin to provide comprehensive anti-spam tagging for an entire site. Typically uses also includes using Qmail-Scanner as a "front end" for Enterprise mail servers such as Notes and Exchange. Qmail-Scanner does all the dirty work - (hopefully) leaving nothing but clean mail for the backend :-)
• Auto-detects email from "postmaster"-style and mailing-list addresses - and doesn't send virus alert reports to them (i.e. attempts to act more like a responsible net citizen)
• Due to the fact that over 99.9% of all email-borne viruses are now sent using forged sender information, Q-S defaults to NOT alerting the sender that a message has been quarantined, unless it was due to a Policy/Perlscan block. This can be turned back to the "old" style by using "--notify sender" instead of the newer default of "--notify psender" (i.e. only notify sender for policy blocks)
• Knows of the virii which forge the From headers - so that the virus appears to come from some poor innocent. Qmail-Scanner will not send alerts to the sender for those types of virii. As the default is to not notify anyway, this only really takes effect if you are using the "--notify sender" option.
• Each message is tagged via a new Received: header with a virus report showing whether it is clean or not and virus scanner version numbers/etc
• [disabled by default] Messages classified as "serious SPAM" by the "--sa-quarantine" option (basically having a really high SA score) will be quarantined off into a "maildir" mail folder (./spam/). This separation into its own maildir allows sites to come up with their own methods of handling false positives. However...
• the "-z" cleanup option will delete messages in the quarantine subfolders older than 14 days - to ensure it doesn't grow too large. If you want to keep them longer, simply script something to move them out daily to another directory/maildir. There is a logrotate script in the contrib directory to automate this (for those systems that can use it - like Redhat/CentOS)
• Can optionally add a descriptive header: X-Qmail-Scannerto every email that passes through the system to allow users to see that a scanner has run over their messages.
• Messages caught by Qmail-Scanner generate an email message (currently supports English, Italian, Afrikaans, Polish, Swedish, Czech, German, Spanish, Turkish, Lithuanian, French, Portuguese, Dutch and Chinese messages) to a configurable combination of the sender, recipients and a "quarantine-admin" address explaining why their message was blocked.
• Can archive some or all processed email (that wasn't quarantined) into an archive maildir. Useful when debugging email-based apps, for backup purposes, and for audit policy reasons. Currently the mail envelope headers (the "rcpt to:" and "mail from:" headers) are appended to the bottom of each message. This option supports being called with a regular expression in which case only envelope headers that match the expression are archived (e.g. can archive "(support|sales)@domain.name" instead of all email)
• If an organization is using clamav, Qmail-Scanner can be directly used for Data Loss Prevention (DLP). Localized clamav signature rules can be written that enable Qmail-Scanner to detect and block emails that clamav detects as "malware". A bit of a misuse perhaps - but clamav's built-in support for archival formats and understanding of document types makes it perfect in this role. If you want Qmail-Scanner to log but not block such DLP "hits" (perhaps because the false positive rates are too high to go with full block-mode), then Qmail-Scanner has a "dlp-monitor" option which tells it which regex of normally quarantinable events are in fact to be let past (i.e. without blocking). It will archive a copy of such messages, and the logging will reflect this was a "DLP:" event.
• Reports via syslog or to a file, a one-line description of each processed message, giving extensive information such as subject line, attachment filenames, sizes, etc.
• Redundant scanning. Not only does it unpack each message before running the scanners over it, it can also scan the original "raw" email message as well as the unpacked components (i.e. if you think a particular scanner has better internal MIME parsing than Qmail-scanner)
• Reporting: in the contrib directory there's qs2mrtg.pl. A perl script for monitoring your syslog files for qmail-scanner records. It then graphs how Qmail-Scanner is processing your emails. It creates different graphs for incoming vs outgoing email, as well as the flow of spam and viruses.

Download

The latest release is 2.11 (via http), and is kindly housed by SourceForge. GnuPG signature of qmail-scanner-2.11.tgz.asc is also available. Of course, you'll be needing my GPG Public Key to verify that.

Requirements

• Netqmail 1.05 (or qmail-1.03 with patches)
• Create a separate account under which to run Qmail-Scanner: defaults to username and groupname "qscand". For extra security, create it with a normal home directory (e.g. "/home/qscand"), but with a "fake" shell (e.g. "/bin/false") - as it's never logged into directly.
• daemontools-0.76+
• reformime from Maildrop 1.3.8+
• Perl 5.005_03+
• Perl module Time::HiRes
• Perl module DB_File (most distributions come with it pre-installed, although the latest Perl doesn't)
• Perl module Sys::Syslog (most distributions come with it pre-installed)
• Perl module MIME::Base64 (most distributions come with it pre-installed)
• Optional: Mark Simpson's TNEF unpacker. Can decode those annoying MS-TNEF MIME attachments that Microsoft mail servers just love to use. If you don't have this, there are several classes of email that Qmail-Scanner basically won't be able to extract attachments in. However, your AV might very well be able to handle them
• Optional: uudecode (part of sharutils on Redhat-style systems)
• Optional: unzip

Patches

Qmail-Scanner relies on Bruce Guenter's QMAILQUEUEpatch to enable qmail-1.03 to call a different qmail-queue program than the one compiled in by default. If you are using netqmail-1.05, then you already have the patch.

Qmail-scanner'sqmail-scanner-queue.plperl script is used instead of Qmail'sqmail-queue binary. Afterqmail-scanner-queue.plhas run, it calls the original qmail-queue binary to resubmit the message back into the system.

• Note 1: Please see www.qmail.org for packaged distributions if you don't feel comfortable building Qmail itself from scratch. There are several RPM-styled src.rpms's available (e.g. QmailToaster, or QmailRocks), along with all sorts of patches you may wish to have in your particular build. As mentioned above, please ensure you have Qmail working before installing Qmail-Scanner. On the Qmail-Scanner mailing-lists, we are seeing too many cases of users having "problems" - which end up being Qmail configuration/understanding issues...
• Note 2: All beginner Qmail Administrators should read Life with Qmail before even attempting to use Qmail

Supported Virus Scanners

The following virus scanners are known to work with qmail-scanner. Remember that only the current releases of scanners are supported. There is little point in running an old scanner - you miss too many viruses.

• Clam Anti-Virus - an Open Source anti-virus scanner
• Spam Assassin Daemon (see FAQ for details)
• Trend's InterScan VirusWall Virus scanner
• AVAST! "avastlite/avastd" virus scanner
• Sophos's "savscan" virus scanner
• H+BEDV's antivir scanner
• Kaspersky's AVPLinux scanner
• McAfee's virus scanner
• Command's virus scanner
• F-Secure Anti-Virus scanner
• F-Prot Anti-Virus scanner
• InocuLAN Anti-Virus scanner
• BitDefender Linux Edition
• Central Command's Vexira anti-virus scanner
• ESET NOD32 Anti-Virus scanner
• Sophie: Daemon front-end to Sophos Sweep (see FAQ for details)
• Trophie: Daemon front-end to Trend iscan (see FAQ for details)
• AVG Anti-Virus
• Team Cymru Malware Hash Registry (MHR) project (free for non-commercial use ONLY)

Other Unix-based scanners should be simple to add support for.

CHANGES

There is a separate page listing changes that have been made between releases

TODO

There is a separate TODO page.

FAQ

There is a separate FAQ page.

Performance/Resource Usage

Adding content/virus scanning to an email server _will_considerably add to the resource usage of that server. As this "wrapper" is written in perl instead of low-level C, quite a lot of memory and file opens/stats occurs just to get it going. Adding to this the actual scanners (i.e. SpamAssassin and AV) memory and CPU usage and it becomes quite complicated (certainly the debugging info shows that the scanner harness spends more time running the external scanners than it does doing things itself [that is to be expected as they do quite a lot of thinking...]).

As a "rule of thumb" I'd suggest you look at how many simultaneous SMTP sessions you are willing your box to have going at any one point in time. Each SMTP session can invoke up to 'n' different virus scanners (although they run one after the other - not simultaneously) and I'd estimate that leads to around 10-20Mb of memory usage per SMTP session. Thus if your dedicated SMTP host has 1024Mb RAM + 2048Mb swap - that should mean you can handle - well heaps ;-) The scanners cause the CPU to be thrashed while they're running, so I'm making sure for our site that our Qmail servers will only accept up to 40 incoming SMTP sessions at any one time - that way I know the box will handle it. As this leads to an increased memory usage, don't forget Qmail's memory limits will need to be increased to deal with it (set via ulimit or softlimit calls with Qmail system startup scripts).

One thing you should test for is what happens if connectivity between this server and another local SMTP server is down for any length of time (due to failure/power outage). When the link is restored, can your server handle the other trying to dump 1,000's of email msgs onto it at once? You need to use softlimit and tcpserver's limit options to ensure your box doesn't get killed. Note that this resource issue isn't "a bug" in Qmail-Scanner. The same thing will happen with a pure, untouched Qmail (or any other) system - it will just happen sooner...

After that scare-mongering I should say that I have tested Qmail-Scanner under ridiculously low resource conditions - and it reacts as it should - so at worst your system should start deferring email. Thankfully DJB's layering of programs is such that this is easy to accomplish :-)

Installation

• IMPORTANT: Ensure all anti-virus scanners and/or SpamAssassin are installed and operational before attempting to install Qmail-Scanner. Ensure these products are usable by non-root accounts (some people have had problems with permissions on some AV scanners in the past). Remember Qmail-Scanner runs as username qscand, so the AV and SpamAssassin must be able to read files created by it - which basically means they need to run as qscand too
• Unpack Qmail-Scanner and run ./configure --help. This will show you whatoptionsare available to you.
• Run ./configure ... [with your options], it will auto-detect what software is installed on your system, and will generate a script specific to your system. If you don't see any errors reported, then the build is (probably) successful.
• Run ./configure again, this time include "--install" along with the options you chose, this will do the same as the previous line, but will also create the directory structure required, and install qmail-scanner-queue.pl
• If you want to manually install it, see the Manual Installation page.
• Before going any further, you can test the installation by running ./contrib/test_installation.sh. This will send four emails: one normal, two "infected" with the EICAR test virus, and one obvious SPAM - to what you set to be the Qmail-Scanner "admin" address. Obviously Qmail-Scanner should let one through,catch the viruses, and tag the SPAM as "spammy" (if SpamAssassin is installed of course!). As Qmail-Scanner now defaults to not notifying anyone when a virus is caught, you may have to depend on the logs (e.g. syslog if you used "--log-details=syslog") to see what Qmail-Scanner did.
  (e.g. tail -10000 /var/log/maillog|egrep " qmail-scanner\[.* Qmail-Scanner_") At this stage qmail-smtpd will need to be "told" that Qmail knows to use qmail-scanner-queue.pl instead of qmail-queue. This is done via the tcpserver control files for SMTP. Look to see where tcpserver for qmail-smtpd gets its rules from - it's the file after the "-x" option (well, that's the CDB version actually - find the text file yourself! ;-). Edit that file and tell qmail-smtpd which IP address ranges (corresponds to SMTP client IP addresses) you want Qmail-Scanner to be invoked on - typically all of them.

#/etc/tcpserver/smtp.rules #

No Qmail-Scanner at all for mail from 127.0.0.1

127.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-queue"

Use Qmail-Scanner without SpamAssassin on any mail from the local network

[it triggers SpamAssassin via the presence of the RELAYCLIENT var]

10.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl" #

Use Qmail-Scanner with SpamAssassin on any mail from the rest of the world

:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

The above example means from now on all SMTP mail will be scanned, but with different characteristics. Mail from the LAN (10. network) will be scanned by the supported virus scanners, whereas mail from the Internet will be scanned for virii AND tagged by SpamAssassin. This finer control allows you a lot of versatility, e.g. virus scanning only performed on mail coming from your Exchange server, and not from your Unix mailing-list servers.

You must increase the amount of memory your system allows qmail-smtpd to run with, as it it now running the entire perl interpreter PLUS virus scanners. Typical installs of Qmail have system rc/startup scripts (e.g. /etc/rc.d/init.d/qmail or /service/smtp/run) that limit the amount of RAM qmail-smtpd can use via ulimit or softlimit. You must increase that to around 20Mb (totally dependent on your OS and choice of anti-virus scanner). If you don't qmail-smtpd will crash with a "qq" error on the receipt of the very first message... The actual amount is dependent on the OS in question as well as the virus scanners being used, so be prepared to experiment a little. Whatever you do, don't just set it to something stupid like 100M "just to be sure". The whole point about limiting RAM usage is so that "unusual" mail messages (e.g. from spammers or hackers) can't cause your system to become unusable by making it run out of RAM. THIS IS A FEATURE OF QMAIL - NOT A BUG. Also note that as the pattern files for any given non-daemonized AV are upgraded, they catch more and more viruses - and as such need more RAM to load into. So your memory sizes will increase with time. This is a good reason to try to stick to daemonized AV products if you can - as the daemon's memory constraints don't impact Qmail-Scanner (i.e. you can reduce the amount of RAM Qmail-Scanner needs by exclusively using daemonized AV)

To scan all mail sent by local shell users, the QMAILQUEUEwill also need to be defined within /etc/profile or the like so that when they send mail, it will be affected as well. Similarly, if you are running Webmail apps, that environment variable will need to be available from within the Web server for Qmail-Scanner to scan any emails sent.

Also, think twice before running Qmail-Scanner in front of any mailing-list servers. Do you really think it's a good idea to have 10,000 messages banging away at your anti-virus system at the same time? Either put your mailing-list servers beyond the reach of your Qmail-Scanner servers, or put the mailing-list on the Qmail-Scanner servers themselves - that way each message is only scanned once and the load issues disappear.

If "$DEBUG=1" (the default) is set within qmail-scanner-queue.pl, then every transaction will be logged to**/var/spool/qscan/qmail-queue.log** - so you'll see how it goes. Regardless of debugging, errors (and attachment info if enabled) should also be recorded in the qmail logs (probably via syslog) - just look for entries containing the string "X-Qmail-Scanner".

Any SMTP sessions that are dropped (due to network outages/etc) may lead to files lying around in /var/spool/qscan . Running**/var/qmail/bin/qmail-scanner-queue.pl -z** at least once daily will ensure such files are deleted when they're over 30 hours old - make a cronjob to do that (see contrib/ for a logrotate script). Also realize that /var/spool/qscan/qmail-queue.log will grow without bounds. At some stage turn debugging off ($DEBUG=0) and delete the logfile. Personally, I like the logfile, so I run a cronjob that just does "mv -f qmail-queue.log qmail-queue.log.1" at 3am every morning. That way logs don't grow without bound, but you still end up with the logs from the past two days. The file can be safely deleted at any time if it becomes a disk-hog, but unless "$DEBUG=0" is set, it'll just get re-created the next time a message comes through (again, the contrib/ logrotate script can be used to take care of this).

Qmail-Scanner contains an internal scanner which allows you to quarantine email based on attachment filenames and/or email headers. Read the minimal document on it for details.

Philosophy behind Quarantining...

When Qmail-Scanner decided to quarantine a message, it moves it into mail folders (maildir format) under /var/spool/qscan/quarantine/. They are split into three different maildir folders based on whether Qmail-Scanner thinks they are a virus, a policy block or high-scoring spam (if you have enabled that option). This means the message can be read in its pure "adulterated" state (e.g. still containing virii/etc) by maildir clients like mutt - or via IMAP (if maildir format supported - you'll have to work that out for yourself). At worse you can just read it with an editor - it's just a MIME text file...

If you want a good IMAP server that supports maildir natively - try Courier-IMAP.

I made the decision to write it into maildir format for performance and reliability reasons - and it expressly makes it difficult for any Windows admin to click on it with their vulnerable Windows mailer and read it :-) Qmail actually comes with a program called /var/qmail/bin/maildir2mbox which can do just that... (you could run it from cron to automatically suck all the new mail messages from **/var/spool/qscan/quarantine/*/new/**into a mbox.)

Also note that Qmail-Scanner only quarantines. It doesn't "clean" messages.

Also this event is logged in /var/spool/qscan/quarantine.login a tab-delimited format (for post-processing). See QSS for an example of one way of generating stats.

If Qmail-Scanner was configured with the "--log-details" option, then a one-line summary of every message processed is recorded either in mailstats.csv or via syslog. e.g:

Aug 14 16:22:41 srvname qmail-scanner[30802]: Clear:RC:1(1.2.3.4): 0.030769 11569 root@x.y jdoe@y.z More_Power! 20020814042234.27902.qmail@x.y 1029298961.30804-0.srvname:10649 Aug 14 16:23:17 srvname qmail-scanner[30820]: Clear:RC:0(1.2.3.4): 0.033618 2021 root@x.y jdoe@y.z Cron_root@x.yrun-parts/etc/cron.daily 20020814042243.28092.qmail@x.y 1029298997.30822-0.srvname:895 Aug 14 16:23:17 srvname qmail-scanner[15885]: Clear:RC:0(1.5.4.3):SA:0(3.0/5.0):CR:PGP(old-signed): 4.66578 5549 fedora-devel-list-bounces@redhat.com Jason.Haar@trimble.co.nz Re:_RFC:_Soname_in_rpm_name 20050128003606.GD16634@neu.nirvana 1106872589.15888-0.mailsrv2.trimble.co.nz:1046 1106872589.15888-1.mailsrv2.trimble.co.nz:189 1106872589.15888-2.mailsrv2.trimble.co.nz:120 orig-mailsrv2.trimble.co.nz110687258948815885:5549

/root@x.y

The format is as follows:

• [standard syslog stuff]
• qmail-scanner[PID] - thus a single SMTP session with 5 recipients would generate 5 syslog entries - but the PID would be the same - showing you it was the same transaction
• message status: "Clear" or description of quarantine event
  • different components of the status are colon-separated, allowing a variety of information to be portrayed in the one record
  • SpamAssassin is recorded as "SA:1" when SA tags the message as Spam, and "SA:0" otherwise
  • The "RC:" bit refers to whether or not the email came from a RELAYCLIENT or not: i.e. "1" says the message was from a "local" SMTP client and "0" means it was from an Internet one. The IP address of the SMTP client is then shown in brackets (127.0.0.1 is used if there was no IP address). This is extremely useful if (say) you want to trigger a pager alert if a local SMTP client sends a virus...
  • If you have "--log-crypto" enabled, "CR:OPTION-XXX" will appear in the log record of any message that uses crypto. Options are:
    * "PGP(signed)" or "PGP(encrypted)" for digitally signed or encrypted PGP/MIME emails. There is also "old-signed"/etc to cover the "older" method of doing PGP
    * "SMIME(signed)" or "SMIME(encrypted)" for digitally signed or encrypted S/MIME emails
    * "CR:ZIP(encrypted)" for password-protected ZIP files
    * "DomKeys" for emails signed with Domain Keys. A bit of a stretch as far as crypto goes - but a signed message is a signed message...
• time taken (sec) to process the message
• "raw" size of message
• envelope sender (i.e. "mail from")
• envelop recipient (i.e. "rcpt to")
• Subject: header
• Message-ID: header
• space-delimited listing of attachment filenames plus their individual sizes appended to the filename

Note: fields are normalized (i.e. converted from base64/Q-P back to glorious 8bit), space-delimited and limited to 1024 chars when syslog is used (with spaces within fields replaced by underscores), and tab-delimited in mailstats.cvs format

Support

This software is released under the GPL as found in the file enclosed.

This package is housed on SourceForge.

Any questions, suggestions, etc must be sent to the mailing-list set up to discuss this, subscribe via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-general , or subscribe to the announcements-only list via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-announce.

Last Updated: