Getting Started With Procmail

Version 0.1d Beta

by Catherine A. Hampton

Updated January 20, 1998

Copyright (C) 1996-1998 by Catherine A. Hampton. If you abide by the Free Software Foundation's COPYING principles with this document and the spam software and forms, you're home free, but don't try to copyright it yourself or sell this information.

Table of Contents

• Introduction
• Setting Up Procmail
• Setting Up the Spam Bouncer
• Invoking Procmail on your Account
• Hints for Users at Specific ISPs
• A2i Communications (rahul.net)
• Best Internet Communications (best.com)
• Netcom Online Communication Services (netcom.com)
• What Should Happen
• Troubleshooting
• Further Information About Procmail
• Conclusion and Acknowledgments

Introduction

This is a quick tutorial intended to get a Procmail neophyte started using Procmail with as little trouble and fuss as possible. It's an expanded and modified version of something I originally posted to an internal newsgroup on my ISP, after yet another user who didn't know Procmail, but needed to learn how to use it so he could filter out spam from his mailbox, asked for help.

Please note -- this is not a tutorial on writing your own Procmail configuration instructions, or recipes. It is just a beginning set of instructions and basic configuration file which should work for most people on most systems. It is particularly designed for users who want to install the Spam Bouncer and/or other Procmail-based spam filters, and who are either totally unfamilar with Procmail or are just beginners.

Return to Table of Contents

Setting Up Procmail

Before You Begin

To set up Procmail, you will need to do some work while telnetted into your shell account. If you do not ever telnet into a shell account and have no idea how to use Unix or a command line, you are probably better off not using Procmail, or getting someone else to set it up for you. In my description below I assume you know these things:

1. How to access your shell account through telnet or rlogin.
2. How to list the contents of a directory.
3. How to copy and delete files from your shell account.
4. How to create a new directory, or remove an old directory.
5. How to use a shell-based text editor (like vi, pico, emacs, or jove) to create, modify and save text files.

Before you retrieve and start customizing the attached Procmail configuration file, you will need to do these things:

1. Make sure you have a directory named "Mail" off of your home directory on your shell account. If you don't, create it. (Please note that Unix is case sensitive -- this must be "Mail" and not "mail" or "MAIL".)
2. Create a directory named "Backup" off of your Mail directory.

What Procmail Will Do For You

The generic Procmail configuration file (or .procmailrc file, as it will be called when you're finished installing it) does the following things:

• Sets the appropriate variables
• Nukes duplicate messages (messages with the same Message ID).
• Creates a backup cache of messages so that, if anything goes wrong, you always have the last 100 messages you received.
• Filter out Mailer_Daemon, root and system administrator mail into a folder of its own.
• Filter out mailing list mail into a "Bulk Mail" folder.
• Filter the remaining mail for spam, either deleting it or putting it into a separate folder, as you prefer.
• Deliver the filtered personal mail to your inbox, where you can read it with Eudora or a POP program, or on shell with elm or pine.

Retrieving the Sample Configuration File

You may retrieve the sample Procmail file named procmail.rc here using your WWW browser, or you may ftp it. The advantage of ftp is that it ensures that the file format will be right. Often, when you retrieve a text file using a WWW browser and then save it to your hard disk, the browser reformats the file. This type of reformatting can break configuration files like the Procmail configuration file.

To ftp procmail.rc, you must do this:

1. Log on to your shell account, and type "cd" to be sure you are in your home directory.
2. Type, "ftp shell3.ba.best.com", and press <Enter>.
3. When ftp prompts you to login, type "anonymous", press <Enter>, and then when prompted for your password, type your email address, and press <Enter> again. (This will log you in.)
4. When ftp gives you a prompt, type "cd /pub/ariel/nospam" and press <Enter>. (This will take you to the folder where the files are stored.)
5. When your prompt returns, type "ascii" and press <Enter>. (This ensures that the line endings on the file will transfer correctly to all systems.)
6. When your prompt returns, type "get procmail.rc" and press <Enter> to retrieve the sample procmail.rc file.
7. When your prompt returns, type "bye" and press <Enter> to end your ftp session.

Configuring procmail.rc

After you've ended your ftp session and returned to the shell prompt in your home directory, you should edit procmail.rc in your favorite unix text editor. If you look through procmail.rc once before you start editing, you will notice it has five sections:

# SECTION 1: SET YOUR VARIABLES
# SECTION 2: GENERAL PREPROCESSING RECIPES
# SECTION 3: MAILING LISTS
# SECTION 4: CALL THE SPAM BOUNCER
# SECTION 5: FINISH DELIVERING THE MAIL

You will need to edit only the first section, to set your variables to values appropriate for your system.

First, please note that anything on a line after a pound sign # is a comment. This includes the Section headings, which Procmail doesn't need or care about, but which make it easier for you to find things in the file. Any line with a ":0" on it (with or without a following colon : or any other letters or numbers) is the beginning of a recipe, a set of Procmail instructions to perform one specific task. Anything on a line following an asterisk * is a condition, a formula which evaluates an individual piece of mail to see if it meets certain criteria. Anything on a line which is not preceeded by a pound sign #, a colon : or an asterisk * is probably a disposition, which either delivers the mail to its destination, forwards it, or (in some cases) deletes it.

Now, look at the variables in Section 1. They are set by default to the values appropriate for my ISP, Best Internet (best.com). If your account is at Best, you're in luck and will have less to do. :) If it is on another ISP, you will have to check the section titled Hints for Users at Specific ISPs to see if we know the right values for your system, or talk with your system administrator to verify what these settings should be.

SHELL=/bin/sh

This specifies the unix shell used to run Procmail. Be sure this points to your system's copy of sh. DO NOT substitute a different shell unless you really know unix.

PATH=$HOME/bin:/bin:/usr/bin:/usr/local/bin

Path for Procmail to use to find the programs it uses and which are called in your procmail.rc configuration. Most of you can probably safely leave this alone -- the basic programs used by Procmail are stored in standard directories on most kinds of Unix.

MAILDIR=$HOME/Mail

The directory where you unix shell mail software stores you mail folders. If you followed instructions earlier, you will have either verified that this exists or created it. Please note that POP mail users also need to designate a MAILDIR.

DEFAULT=$HOME/inbox

The default incoming mail box to which your unix system delivers mail. POP users MUST set this to the proper directory and file name or their POP mail client will not be able to find their mail. Hints for Users at Specific ISPs has the correct settings for a number of large ISPs. If the correct setting is not there, please ask you System Administrator for the right setting.

Users who read their mail on their unix shell accounts should designate the folder which their preferred mail program uses as its incoming mail folder.

LOGFILE=$MAILDIR/log

Keeps a log of what Procmail does with each incoming email message. Recommended, so that if something goes wrong, there's a record of what Procmail was doing.

SENDMAIL=/usr/sbin/sendmail

Tells Procmail where to find your system's copy of sendmail, the program used to autoreply to messages. If the Hints for Users at Specific ISPs doesn't have this information for your system, ask your System Administrator.

FORMAIL=/usr/local/bin/formail

Tells Procmail where to find your system's copy of formail, a program which is part of the Procmail package and which is used to modify message headers and reformat messages before delivering them to their ultimate destination. If the Hints for Users at Specific ISPs doesn't have this information for your system, ask your System Administrator.

SBDIR=$HOME/shared

Directory where Spam Bouncer files are located. If you prefer a different directory name, change this to point to the directory where you will be keeping the Spam Bouncer program files.

ADMINFOLDER=$MAILDIR/admin.incoming

File where bounced mail, and mail from adminstrative accounts (such as root, postmaster, or abuse) is stored. POP users will probably want to set this to $DEFAULT, and let their POP client's mail filtering handle the sorting of their mail. Shell account users can set this to any file they want.

BLOCKFOLDER=$MAILDIR/block.incoming

File for suspicious mail which is probably spam, but which may not be. POP users will probably want to set this to $DEFAULT, and let their POP client's mail filtering put it in a separate folder for checking, just in case something genuine got caught with the spam. Shell account users can set this to any file they want. I do not recommend setting this to /dev/null because it is possible this may contain legitimate mail.

BULKFOLDER=$MAILDIR/bulk.incoming

File for bulk mail which appears legitimate, but which you don't want cluttering up your incoming mail folder. This is for mailing list mail and similar types of mail. POP users will probably want to set this to $DEFAULT, and let their POP client's mail filtering put it in a separate folder. Shell account users can set this to any file they want.

SPAMFOLDER=$MAILDIR/spam.incoming

File for mail which is definitely spam. You can set SPAMFOLDER=/dev/null to delete spam entirely, which I recommend for most users. This will save POP mail users the time and trouble of downloading. POP users who really want to look at their spam (perhaps to complain or to learn more about understanding forged mail headers) can set this to $DEFAULT, and let their POP client's mail filtering put it in an appropriate folder. Shell users who want to see the stuff can designate any folder they wish.

When you are finished setting up Procmailrc as you like it, type "mv procmail.rc .procmailrc" and press <Enter>.

Return to Table of Contents

Setting Up the Spam Bouncer

Next, BEFORE YOU INSTALL THE SPAM BOUNCER, read the Spam Bouncer WWW page.

When you have read it, your next step should be to create a directory off of your home directory called "shared", or whatever you set up as your SBDIR in your .procmailrc file. Retrieve "sb.rc" and its associated files from the Spam Bouncer page and save it in your shared directory. You don't need to customize it.

On certain systems there will already be a shared installation of the Spam Bouncer. Check below for information about any such set-up on your ISP.

Return to Table of Contents

Invoking Procmail on your Account

Finally, unless you are a Linux user, you will need to create an appropriate .forward file in your home directory to forward your mail to Procmail for filtering. (Linux users are lucky -- Linux automatically looks for a .procmailrc file in each user's home directory and automatically invokes Procmail if it finds one.) If you have an account at Best Internet, this is the .forward you need to use:

"|IFS=' '&&exec /usr/local/bin/procmail -f-||exit 75 #login"

You need to type that exactly as shown, including the quote marks on either end. BE SURE TO SUBSTITUTE YOUR LOGIN NAME for "login". This sample .forward file is probably similar to what you need, but the .forward file must be exactly right, or it won't work. If you are not at Best Internet, check with your system administrator or another, more experienced Procmail user on your system.

Return to Table of Contents

Hints for Users at Specific ISPs

If your ISP is not listed here, and you know the following information for your ISP, please let me know so that I can add an entry for your ISP.

ISPs without Shell Access

If you have an account at one of the following ISPs, you do not have access to a Unix shell account or Procmail from that account. If you need to use Procmail, you will need to get an account on a system which allows Unix shell access and has Procmail installed and available for users.

Angelfire (angelfire.com)
AOL (aol.com)
Bigfoot (bigfoot.com)
Compuserve (compuserve.com)
GEnie (genie.com)
Hotmail (hotmail.com)
Juno (juno.com)
Microsoft Network (msn.com)
Netcom Netcruiser/Netcomplete (ix.netcom.com)
Prodigy (prodigy.com/prodigy.net)
USA.net (usa.net)
Yahoo (yahoo.com)

Return to Table of Contents

A2i (rahul.net)

System Settings and Information

Notes about Special Features and Issues

Please note that a2i's standard Procmail version is quite old, v2.81, but version 3.11pre4 has also been installed. This Procmail recipe has not been tested with the earlier version of Procmail and may fail -- please use the newer version.

A2i users who want to use rahul.net's nojunk filtering (which I highly recommend -- it is better and more sophisticated than the Spam Bouncer) will want to invoke Procmail though a .nojunk.patterns file in their home directories instead of through a .forward file. Here is a simple .nojunk.patterns file to run Procmail after nojunk filtering:

global
logheaders
pipe = /etc/LOCAL/bin/Procmail3.11p4 -Yf-

Please note that you use .nojunk.patterns INSTEAD OF a .forward file. If you create a .forward file in your home directory, this turns off all nojunk filtering and sends your unfiltered email to whatever destination you specify.

If you do this, you can delete the line which calls the Spam Bouncer from procmail.rc and not install it. While it is perfectly possible to run the Spam Bouncer concurrently with nojunk filtering, nojunk filtering so rarely fails that it probably isn't worth the trouble.

Return to Table of Contents

Best Internet Services (best.com)

System Settings and Information

Please substitute your shell login name for "login" in the .forward setting.

Notes about Special Features and Issues

If you have an account on shell3, you can issue this command while logged into your shell account and in your home directory:

ln -s ~ariel/shared ~/shared

This will create a symbolic link from my shared subdirectory, which contains the latest tested version of the Spam Bouncer, to a subdirectory off of your home directory, called "shared". The advantage to doing this is that I update the Spam Bouncer on an average three to five times a week with new spam sites. This way, you'll always be using the latest version and won't have to do anything to stay current.

If someone with an account on another Best shell machine would like to make the Spam Bouncer available to everyone on those machines and will take on the responsibility of keeping it up to date, email me and we'll make arrangements. :)

Return to Table of Contents

Netcom Online Communications Services (netcom.com)

System Settings and Information

Please note that these instructions apply only to Netcom shell accounts. Netcomplete/Netcruiser account holders do not have shell access and cannot use Procmail.

Please substitute your shell login name for "login" in the .forward setting.

Notes about Special Features and Issues

Netcom's standard Procmail installation is an early version, one that procmail.rc has not been tested with. Please use the newer version in the NUGLOPS (Netcom Users Group List of Programs) directory.

Return to Table of Contents

What Should Happen

Once you create the .forward, you are up and running with Procmail, and have the Spam Bouncer installed to filter your mail for spam. If you are at Best or one of many sites which now block mail from Cyberpromo and other egregious spam sites, the filter should catch any spam from these sites which is missed because the spammer forged the headers and sent his spam through (or "bounced it off") someone else's SMTP server. :/

Depending on how you set up your variables, this setup will create the following files:

• In your home directory, it will create a standard inbox file containing spam-filtered personal mail, which can be retrieved as usual using Eudora, Pegasus Mail, Netscape, Internet Explorer, or on shell using elm or pine. (In some cases, this file may be in a system mail directory rather than your home directory.)

• In your Mail directory, it will create an admin.incoming file which contains any bounces from Mailer Daemons (automated programs), mail from root, from admin, from sysadmin, and from abuse. You can read this mail on shell, using elm or pine to retrieve this folder. If you prefer to read it in your POP mail reader, you can change the folder "admin.incoming" to "$DEFAULT".

• In your Mail directory, it will create a bulk.incoming file of mail from mailing lists and mail which was sent using a blind carbon copy or mailing list, in which none of your personal email addresses appears. You can read this mail on shell, or in your POP mail reader by changing all instances of "bulk.incoming" to "$DEFAULT".

• In your Mail directory, it will create a block.incoming file of mail which the Spam Bouncer trapped using pattern matching, but which did not come from a known spam source and which might possibly be legitimate. You can read this mail on shell, or in your POP mail reader by changing your BLOCKFOLDER variable to "$DEFAULT".

• In your Mail directory, unless you set SPAMFOLDER to /dev/null, it will create a folder called "spam.incoming", with known spam messages. If you want to make a point of learning how to analyze spam and complain about it, you should not set SPAMFOLDER to /dev/null. If you just don't want to be bothered, set SPAMFOLDER to /dev/null.

• In your Mail directory, it will create a file named "log" with a record of all mail you received, who it was From:, the Subject: and what Procmail did with it.

• In your Mail/backup directory, it will maintain copies of the last 100 emails you received, just in case something goes wrong and you need to retrieve your mail. Ask for help if something goes wrong. :)

Return to Table of Contents

Further Information About Procmail

Those who want to learn how to write Procmail recipes of their own and customize their filters to their own needs should:

1. Read the Procmail man pages from your shell prompt. They are:
• Procmail (program overview)
• Procmailrc (rules for .procmailrc)
• Procmailex (examples of .procmailrc recipes)
• Procmailsc (how to use Procmail's scoring feature)
2. Join the Procmail mailing list, by sending email to Procmail-request@Informatik.RWTH-Aachen.DE, with anything in the Subject: line, and a body that reads "subscribe".

Return to Table of Contents

Troubleshooting

Procmail is a powerful, and therefore dangerous, program. I recommend installing it for the first time when you have a few hours to spend keeping an eye on your system, to make sure everything is working right. I'll write this section later. For now, when installing try to have a friend who knows Procmail available to help out. If anything goes wrong, turn off Procmail by renaming the .forward file to new.forward.

The last 100 pieces of your incoming email will be stored in $MAILDIR/backup. To retrieve them, you can change to that directory, and issue this command:

cat msg* >>! newmail

You can then either issue a similar command to append this mail to your mailbox, or read it on your shell system using elm or pine.

Return to Table of Contents

Conclusion and Acknowledgments

Let me know if this is helpful, or if some part of the explanation isn't clear, and I'll fix the bugs and credit those who helped. :)

I would also like to get the following information from users on systems with shell access and Procmail which are not listed in the "Specific Instructions" section:

1. Location of users' incoming mailboxes.
2. Locations of ksh, sendmail, and formail on your system.
3. Version of Procmail running on your system. (You can get this by typing "Procmail -v" from your shell command line.)
4. Correct syntax for .forward file to invoke Procmail on your system.

In later versions of this document I will include this information to help users, and system administrators who would rather not be bothered with routine questions all the time. ;>

Thank you!

Return to Table of Contents

The Spam Bouncer | Home Page | Send me email

©1996-1999 by Catherine A. Hampton <ariel@tempest.boxmail.com>. All rights reserved.