qms-analog: Qmail-Scanner Log File Analyzer ------------------------------------------- Version: 0.4.4, 06/18/2005 Distribution Files ------------------ COPYING - The GPL Version 2 License file Makefile - builds the qms-analog utility qmail-scanner-1.25-st-qms-20050618.patch - patch file for clean qmail-scanner-1.25 distro which contains the qms event logger and the popular "st" patch qms-analog-types.txt - defines the event log types provided by the patch qms-config - (patch) qmail-scanner config script for qms-analog qms-config-cwrapper - (patch) qmail-scanner config script for qms-analog used when perl doesn't have setuid support qms-config-monitor - (patch) qmail-scanner config script for qms-analog and qms-monitor qms-config-monitor-cwrapper - (patch) qmail-scanner config script for qms-analog and qms-monitor used when perl doesn't have setuid support qms-monitor-move.sh - (patch) example email moving script to be used with cron or manually README - this file RELEASE-NOTES - version change log src/ - source directory for qms-analog What You Get ------------ /var/qmail/bin/qms-analog The utility which takes /var/spool/qmailscan/qms-events.log records as input from stdin and generates statistics on stdout. /var/qmail/bin/qmail-scanner-queue.pl The patched version which generates nice logs in /var/spool/qmailscan/qms-events.log and optionally provides account monitoring if qms-monitor is enabled !!!!!!!! ATTENTION: READ THIS FIRST !!!!!!!! -------------------------------------------- qms-analog requires a patch be applied to your qmail-scanner-1.25 distribution in order to generate a new, more legible log file. The patch file includes the popular "st" patch which adds useful capabilities to qmail-scanner. The following are minimum requirements for qms-analog to work: 1) qmail-scanner version 1.25 (unpatched, clean distro) 2) ClamAV or other qmail-scanner supported AV software. 3) Spamassassin If you don't have these, and are unwilling to upgrade or install them, DO NOT USE qms-analog. I cannot be responsible for what might happen. Disclaimer: ----------- Generally speaking, it is a quite simple matter to upgrade versions of qms-analog. Patch a clean distro of qmail-scanner-1.25, configure it, modify qmail-scanner-queue.pl if using the C wrapper instead of setuid, then build and install qms-analog. I have done this time and time again with no adverse effects on my Qmail installation or any of the ancillary utilities that were installed in the QmailRocks procedure. Having said that, as with all open-source software, no guarantee is expressed or implied in any way, and I am not responsible for mistakes or abnormalities in your particular installation. If you meet these requirements, let's get started... Note Concerning Where This Fits In the QmailRocks Procedure ----------------------------------------------------------- qms-analog is now part of the QmailRocks procedure. See http://qmailrocks.org for details. A. Patching the qmail-scanner-1.25 Distribution ----------------------------------------------- 1) Obtain the unpatched source distribution qmail-scanner-1.25.tgz. 2) Extract it to the location of your choice. 3) Make a backup copy of the qmail-scanner-1.25 directory before patching it: cp -R qmail-scanner-1.25 qmail-scanner-1.25-orig 4) Copy qmail-scanner-1.25-st-qms-20050618.patch from the qms-analog distro to the qmail-scanner-1.25 directory where the tarball was extracted. cp qmail-scanner-1.25-st-qms-20050618.patch 5) Change directory to the qmail-scanner-1.25 distribution: cd 6) Patch qmail-scanner-1.25: patch -p1 < qmail-scanner-1.25-st-qms-20050618.patch 7) Configure qmail-scanner-1.25: To configure qms-monitor support: CONFFILE = qms-config-monitor To disable qms-monitor support: CONFFILE = qms-config Substitute the appropriate config file name above for the place holder CONFFILE in the following directions. a) Edit CONFFILE to insert your domain name, postmaster account name, and local domain list. Also modify other settings of interest including qms-monitor accounts, etc. b) Make the script executable: chmod 755 ./CONFFILE c) Execute: ./CONFFILE d) if the test configure looks good, install it: ./CONFFILE install Proceed to step 8. e) if the script complains about setuid, execute: ./CONFFILE-cwrapper f) if that works, execute: ./CONFFILE-cwrapper install to install it and follow the "C Wrapper" instructions This sets up qmail-scanner in a qms-analog friendly way. Note: I changed debug to default to disabled in the patch. You can add "--debug=1" as an option to configure to enable it. It gets very large and is of no real use anyway to users. The st --minidebug together with --sa_alt and --sa_debug produce much better debug output anyway. 8) After successful configuration , if you are using the C-wrapper instead of perl's setuid, follow the directions in the qmail-scanner-1.25/contrib/qmail-scanner-queue.c file - to modify the permissions of qmail-scanner-queue.pl and the perl tag at the top of that file (delete the "-T"). 9) Copy the qmailstats script from the qms-analog distro directory to /var/qmail/bin: cp /qmailstats /var/qmail/bin chmod 0755 /var/qmail/bin/qmailstats 10) The following log files will need to be rotated or otherwise monitored so they do not grow too large: /var/spool/qmailscan/qmail-queue.log /var/spool/qmailscan/qms-events.log B. Building qms-analog and installing it with a new qmailstats script --------------------------------------------------------------------- 1) Become root 2) cd to the qms-analog directory (wherever you extracted it) 3) make all C. Testing ---------- 1) Allow several logs to accumulate in /var/spool/qmailscan/qms-events.log. 2) Execute: cat /var/spool/qmailscan/qms-events.log | /var/qmail/bin/qms-analog 0 This should dump the qms-analog results to stdout (the shell you ran it from). 3) If that looks good, execute: /var/qmail/bin/qmailstats This should generate the nightly email to the postmaster including the qms-analog stats at the bottom. 4) If those two tests pass, you are done! D. Using qms-analog ------------------- 1) qms-analog reads the log records from stdin. Thus you can pipe the output of "cat /var/spool/qmailscan/qms-events.log" into qms-analog. 2) qms-analog writes its results to stdout. This can be redirected to a file or viewed on the controlling console. 3) qms-analog requires the hours-of-history argument which specifies the number of hours of historical stats to compile. You can also pass a second argument, sort-key. This specifies the order of the account based statistics. usage: qms-analog hours-of-history hours-of-history (0 - n) hours of history to collect 0 => all records sort-key (optional) sort key for account statistics msgbw (default) msg bandwidth - successful msgs alpha alphanumeric by account name virus number of viruses received saavg Spamassassin avg score sadet Spamassassin msgs detected Some examples: "qms-analog 24" - use only records within the last 24 hours, sort by msg bandwidth "qms-analog 168" - use only records within the last 7 days, sort by msg bandwidth "qms-analog 0" - use all records, sort by msg bandwidth "qms-analog 0 alpha" - use all records, sort alphabetically "qms-analog 0 saavg" - use all records, sort by Spam average score 4) Although qms-analog is installed as part of the "qmailstats" script, it could easily be invoked from a custom script which could be run from a shell at any time, or as part of the cron daemon's tasks. See "qmailstats" for example usage. 5) Notes on Statistical Output The headings for the account statistics are described below: MsgRx - messages successfully received (not virus, not deleted or quarantined spam) MsgTx - messages successfully transmitted (not virus) %Total - what percent of total successful messages for the mail server (MsgRx + MsgTx) comprises ScanTime - total time in secs that qmail-scanner took to process the messages VirusRx - messages received that were intercepted as containing a virus VirusTx - messages transmited that were intercepted as containing a virus SA-AVG - average Spamassassin score for all messages for this account run through Spamassassin SA-MRK - number of messages for this account marked and delivered by qmail-scanner based on the Spamassassin score SA-DEL - number of messages for this account deleted by qmail-scanner based on the Spamassassin score SA-REJ - number of messages for this account rejected by qmail-scanner based on the Spamassassin score SA-QUA - number of messages for this account quarantined by qmail-scanner based on the Spamassassin score E. Notes -------- If you have any problems, first re-read the directions and make sure you did everything as prescribed, and if you are still having a problem, just restore the original qmail-scanner-1.25 distribution (we backed it up, right?) and configure it as normal. Also, please report problems or suggestions to the Sourceforge mailing list or to the appropriate Sourceforge forum at: http://sourceforge.net/projects/qms-analog/ I try to be very responsive. Mark Teel mteel@users.sourceforge.net