fetchmail - Unix, Linux Command

NAME

SYNOPSIS

DESCRIPTION

The fetchmail program can gather mail from servers supporting any of the common mail-retrieval protocols: POP2 (legacy, to be removed from future release), POP3, IMAP2bis, IMAP4, and IMAP4rev1. It can also use the ESMTP ETRN extension and ODMR. (The RFCs describing all these protocols are listed at the end of this manual page.)

While fetchmail is primarily intended to be used over on-demand TCP/IP links (such as SLIP or PPP connections), it may also be useful as a message transfer agent for sites which refuse for security reasons to permit (sender-initiated) SMTP transactions with sendmail.

If fetchmail is used with a POP or an IMAP server, it has two fundamental modes of operation for each user account from which it retrieves mail: singledrop- and multidrop-mode. In singledrop-mode, fetchmail assumes that all messages in the user’s account are intended for a single recipient. An individual mail message will not be inspected for recipient information, rather, the identity of the recipient will either default to the local user currently executing fetchmail, or else will need to be explicitly specified in the configuration file. Singledrop-mode is used when the fetchmailrc configuration contains at most a single local user specification for a given server account.

With multidrop-mode, fetchmail is not able to assume that there is only a single recipient, but rather that the mail server account actually contains mail intended for any number of different recipients. Therefore, fetchmail must attempt to deduce the proper "envelope recipient" from the mail headers of each message. In this mode of operation, fetchmail almost resembles an MTA, however it is important to note that neither the POP nor IMAP protocols were intended for use in this fashion, and hence envelope information is often not directly available. Instead, fetchmail must resort to a process of informed guess-work in an attempt to discover the true envelope recipient of a message, unless the ISP stores the envelope information in some header (not all do). Even if this information is present in the headers, the process can be error-prone and is dependent upon the specific mail server used for mail retrieval. Multidrop-mode is used when more than one local user is specified for a particular server account in the configuration file. Note that the forgoing discussion of singledrop- and multidrop-modes does not apply to the ESMTP ETRN or ODMR retrieval methods, since they are based upon the SMTP protocol which specifically provides the envelope recipient to fetchmail.

As each message is retrieved, fetchmail normally delivers it via SMTP to port 25 on the machine it is running on (localhost), just as though it were being passed in over a normal TCP/IP link. fetchmail provides the SMTP server with an envelope recipient derived in the manner described previously. The mail will then be delivered locally via your system’s MDA (Mail Delivery Agent, usually sendmail(8) but your system may use a different one such as smail, mmdf, exim, postfix, or qmail). All the delivery-control mechanisms (such as .forward files) normally available through your system MDA and local delivery agents will therefore work automatically.

If no port 25 listener is available, but your fetchmail configuration was told about a reliable local MDA, it will use that MDA for local delivery instead.

If the program fetchmailconf is available, it will assist you in setting up and editing a fetchmailrc configuration. It runs under the X window system and requires that the language Python and the Tk toolkit be present on your system. If you are first setting up fetchmail for single-user mode, it is recommended that you use Novice mode. Expert mode provides complete control of fetchmail configuration, including the multidrop features. In either case, the ’Autoprobe’ button will tell you the most capable protocol a given mailserver supports, and warn you of potential problems with that server.

GENERAL OPERATION

Each server name that you specify following the options on the command line will be queried. If you don’t specify any servers on the command line, each ’poll’ entry in your ~/.fetchmailrc file will be queried.

To facilitate the use of fetchmail in scripts and pipelines, it returns an appropriate exit code upon termination -- see EXIT CODES below.

The following options modify the behavior of fetchmail. It is seldom necessary to specify any of these once you have a working .fetchmailrc file set up.

Almost all options have a corresponding keyword which can be used to declare them in a .fetchmailrc file.

Some special options are not covered here, but are documented instead in sections on AUTHENTICATION and DAEMON MODE which follow.

General Options

Disposal Options

Protocol and Query Options

        openssl x509 -in cert.pem -noout -md5 -fingerprint

Delivery Control Options

        --smtphost server1,server2/2525,server3,/var/imap/socket/lmtp

Resource Limit Control Options

Authentication Options

        interface/iii.iii.iii.iii[/mmm.mmm.mmm.mmm]

Miscellaneous Options

Removed Options

USER AUTHENTICATION AND ENCRYPTION

If the mailserver is a Unix machine on which you have an ordinary user account, your regular login name and password are used with fetchmail. If you use the same login name on both the server and the client machines, you needn’t worry about specifying a user-id with the -u option -- the default behavior is to use your login name on the client machine as the user-id on the server machine. If you use a different login name on the server machine, specify that login name with the -u option. e.g. if your login name is ’jsmith’ on a machine named ’mailgrunt’, you would start fetchmail as follows:

Using netrc files

If you do not specify a password, and fetchmail cannot extract one from your ~/.fetchmailrc file, it will look for a ~/.netrc file in your home directory before requesting one interactively; if an entry matching the mailserver is found in that file, the password will be used. Fetchmail first looks for a match on poll name; if it finds none, it checks for a match on via name. See the ftp(1) man page for details of the syntax of the ~/.netrc file. To show a practical example, a .netrc might look like this:

machine hermes.example.org
login joe
password topsecret

This feature may allow you to avoid duplicating password information in more than one file.

On mailservers that do not provide ordinary user accounts, your user-id and password are usually assigned by the server administrator when you apply for a mailbox on the server. Contact your server administrator if you don’t know the correct user-id and password for your mailbox account.

POP3 VARIANTS

Early versions of POP3 (RFC1081, RFC1225) supported a crude form of independent authentication using the rhosts file on the mailserver side. Under this RPOP variant, a fixed per-user ID equivalent to a password was sent in clear over a link to a reserved port, with the command RPOP rather than PASS to alert the server that it should do special checking. RPOP is supported by fetchmail (you can specify ’protocol RPOP’ to have the program send ’RPOP’ rather than ’PASS’) but its use is strongly discouraged, and support will be removed from a future fetchmail version. This facility was vulnerable to spoofing and was withdrawn in RFC1460.

RFC1460 introduced APOP authentication. In this variant of POP3, you register an APOP password on your server host (on some servers, the program to do this is called popauth(8)). You put the same password in your ~/.fetchmailrc file. Each time fetchmail logs in, it sends an MD5 hash of your password and the server greeting time to the server, which can verify it by checking its authorization database.

Note that APOP is no longer considered resistant against man-in-the-middle attacks.

RETR or TOP

fetchmail will always use the RETR command if "fetchall" is set. fetchmail will also use the RETR command if "keep" is set and "uidl" is unset. Finally, fetchmail will use the RETR command on Maillennium POP3/PROXY servers (used by Comcast) to avoid a deliberate TOP misinterpretation in this server that causes message corruption.

In all other cases, fetchmail will use the TOP command. This implies that in "keep" setups, "uidl" must be set if "TOP" is desired.

Note that this description is true for the current version of fetchmail, but the behavior may change in future versions. In particular, fetchmail may prefer the RETR command because the TOP command causes much grief on some servers and is only optional.

ALTERNATE AUTHENTICATION FORMS

If your fetchmail was built with Kerberos support and you specify Kerberos authentication (either with --auth or the .fetchmailrc option authenticate kerberos_v4) it will try to get a Kerberos ticket from the mailserver at the start of each query. Note: if either the pollname or via name is ’hesiod’, fetchmail will try to use Hesiod to look up the mailserver.

If you use POP3 or IMAP with GSSAPI authentication, fetchmail will expect the server to have RFC1731- or RFC1734-conforming GSSAPI capability, and will use it. Currently this has only been tested over Kerberos V, so you’re expected to already have a ticket-granting ticket. You may pass a username different from your principal name using the standard --user command or by the .fetchmailrc option user.

If your IMAP daemon returns the PREAUTH response in its greeting line, fetchmail will notice this and skip the normal authentication step. This can be useful, e.g. if you start imapd explicitly using ssh. In this case you can declare the authentication value ’ssh’ on that site entry to stop .fetchmail from asking you for a password when it starts up.

If you use client authentication with TLS1 and your IMAP daemon returns the AUTH=EXTERNAL response, fetchmail will notice this and will use the authentication shortcut and will not send the passphrase. In this case you can declare the authentication value ’external’
on that site to stop fetchmail from asking you for a password when it starts up.

If you are using POP3, and the server issues a one-time-password challenge conforming to RFC1938, fetchmail will use your password as a pass phrase to generate the required response. This avoids sending secrets over the net unencrypted.

Compuserve’s RPA authentication is supported. If you compile in the support, fetchmail will try to perform an RPA pass-phrase authentication instead of sending over the password en clair if it detects "@compuserve.com" in the hostname.

If you are using IMAP, Microsoft’s NTLM authentication (used by Microsoft Exchange) is supported. If you compile in the support, fetchmail will try to perform an NTLM authentication (instead of sending over the password en clair) whenever the server returns AUTH=NTLM in its capability response. Specify a user option value that looks like ’user@domain’: the part to the left of the @ will be passed as the username and the part to the right as the NTLM domain.

Secure Socket Layers (SSL) and Transport Layer Security (TLS)

You can access SSL encrypted services by specifying the --ssl option. You can also do this using the "ssl" user option in the .fetchmailrc file. With SSL encryption enabled, queries are initiated over a connection after negotiating an SSL session, and the connection fails if SSL cannot be negotiated. Some services, such as POP3 and IMAP, have different well known ports defined for the SSL encrypted services. The encrypted ports will be selected automatically when SSL is enabled and no explicit port is specified. The --sslproto option can be used to select the SSL protocols (default: v2 or v3). The --sslcertck command line or sslcertck run control file option should be used to force strict certificate checking - see below.

If SSL is not configured, fetchmail will usually opportunistically try to use TLS. TLS can be enforced by using --sslproto "TLS1". TLS connections use the same port as the unencrypted version of the protocol and negotiate TLS via special parameter. The --sslcertck command line or sslcertck run control file option should be used to force strict certificate checking - see below.

--sslcheck recommended: When connecting to an SSL or TLS encrypted server, the server presents a certificate to the client for validation. The certificate is checked to verify that the common name in the certificate matches the name of the server being contacted and that the effective and expiration dates in the certificate indicate that it is currently valid. If any of these checks fail, a warning message is printed, but the connection continues. The server certificate does not need to be signed by any specific Certifying Authority and may be a "self-signed" certificate. If the --sslcertck command line option or sslcertck run control file option is used, fetchmail will instead abort if any of these checks fail. Use of the sslcertck or --sslcertck option is advised.

Some SSL encrypted servers may request a client side certificate. A client side public SSL certificate and private SSL key may be specified. If requested by the server, the client certificate is sent to the server for validation. Some servers may require a valid client certificate and may refuse connections if a certificate is not provided or if the certificate is not valid. Some servers may require client side certificates be signed by a recognized Certifying Authority. The format for the key files and the certificate files is that required by the underlying SSL libraries (OpenSSL in the general case).

A word of care about the use of SSL: While above mentioned setup with self-signed server certificates retrieved over the wires can protect you from a passive eavesdropper, it doesn’t help against an active attacker. It’s clearly an improvement over sending the passwords in clear, but you should be aware that a man-in-the-middle attack is trivially possible (in particular with tools such as dsniff, http://monkey.org/~dugsong/dsniff/). Use of strict certificate checking with a certification authority recognized by server and client, or perhaps of an SSH tunnel (see below for some examples) is preferable if you care seriously about the security of your mailbox and passwords.

ESMTP AUTH

fetchmail also supports authentication to the ESMTP server on the client side according to RFC 2554. You can specify a name/password pair to be used with the keywords ’esmtpname’ and ’esmtppassword’; the former defaults to the username of the calling user.

DAEMON MODE

Introducing the daemon mode

Starting the daemon mode

Example: simply invoking

It is also possible to set a polling interval in your ~/.fetchmailrc file by saying ’set daemon <interval>’, where <interval> is an integer number of seconds. If you do this, fetchmail will always start in daemon mode unless you override it with the command-line option --daemon 0 or -d0.

Only one daemon process is permitted per user; in daemon mode, fetchmail sets up a per-user lockfile to guarantee this. (You can however cheat and set the FETCHMAILHOME environment variable to overcome this setting, but in that case, it is your responsibility to make sure you aren’t polling the same server with two processes at the same time.)

Awakening the background daemon

Normally, calling fetchmail with a daemon in the background sends a wake-up signal to the daemon and quits without output. The background daemon then starts its next poll cycle immediately. The wake-up signal, SIGUSR1, can also be sent manually. The wake-up action also clears any authentication or multiple timeouts.

Terminating the background daemon

The option --quit will kill a running daemon process instead of waking it up (if there is no such process, fetchmail will notify you. If the --quit option appears last on the command line, fetchmail will kill the running daemon process and then quit. Otherwise, fetchmail will first kill a running daemon process and then continue running with the other options.

Useful options for daemon mode

The -L <filename> or --logfile <filename> option (keyword: set logfile) is only effective when fetchmail is detached. This option allows you to redirect status messages into a specified logfile (follow the option with the logfile name). The logfile is opened for append, so previous messages aren’t deleted. This is primarily useful for debugging configurations. Note that fetchmail does not detect if the logfile is rotated, the logfile is only opened once when fetchmail starts. You need to restart fetchmail after rotating the logfile and before compressing it (if applicable).

The --syslog option (keyword: set syslog) allows you to redirect status and error messages emitted to the syslog(3) system daemon if available. Messages are logged with an id of fetchmail, the facility LOG_MAIL, and priorities LOG_ERR, LOG_ALERT or LOG_INFO. This option is intended for logging status and error messages which indicate the status of the daemon and the results while fetching mail from the server(s). Error messages for command line options and parsing the .fetchmailrc file are still written to stderr, or to the specified log file. The --nosyslog option turns off use of syslog(3), assuming it’s turned on in the ~/.fetchmailrc file, or that the -L or --logfile <file> option was used.

The -N or --nodetach option suppresses backgrounding and detachment of the daemon process from its control terminal. This is useful for debugging or when fetchmail runs as the child of a supervisor process such as init(8) or Gerrit Pape’s runit. Note that this also causes the logfile option to be ignored (though perhaps it shouldn’t).

Note that while running in daemon mode polling a POP2 or IMAP2bis server, transient errors (such as DNS failures or sendmail delivery refusals) may force the fetchall option on for the duration of the next polling cycle. This is a robustness feature. It means that if a message is fetched (and thus marked seen by the mailserver) but not delivered locally due to some transient error, it will be re-fetched during the next poll cycle. (The IMAP logic doesn’t delete messages until they’re delivered, so this problem does not arise.)

If you touch or change the ~/.fetchmailrc file while fetchmail is running in daemon mode, this will be detected at the beginning of the next poll cycle. When a changed ~/.fetchmailrc is detected, fetchmail rereads it and restarts from scratch (using exec(2); no state information is retained in the new instance). Note also that if you break the ~/.fetchmailrc file’s syntax, the new instance will softly and silently vanish away on startup.

ADMINISTRATIVE OPTIONS

The --postmaster <name> option (keyword: set postmaster) specifies the last-resort username to which multidrop mail is to be forwarded if no matching local recipient can be found. It is also used as destination of undeliverable mail if the ’bouncemail’ global option is off and additionally for spam-blocked mail if the ’bouncemail’ global option is off and the ’spambounce’ global option is on. This option defaults to the user who invoked fetchmail. If the invoking user is root, then the default of this option is the user ’postmaster’. Setting postmaster to the empty string causes such mail as described above to be discarded - this however is usually a bad idea. See also the description of the ’FETCHMAILUSER’ environment variable in the ENVIRONMENT section below.

The --nobounce behaves like the "set no bouncemail" global option, which see.

The --invisible option (keyword: set invisible) tries to make fetchmail invisible. Normally, fetchmail behaves like any other MTA would -- it generates a Received header into each message describing its place in the chain of transmission, and tells the MTA it forwards to that the mail came from the machine fetchmail itself is running on. If the invisible option is on, the Received header is suppressed and fetchmail tries to spoof the MTA it forwards to into thinking it came directly from the mailserver host.

The --showdots option (keyword: set showdots) forces fetchmail to show progress dots even if the current tty is not stdout (for example logfiles). Fetchmail shows the dots by default when run in nodetach mode or when daemon mode is not enabled.

By specifying the --tracepolls option, you can ask fetchmail to add information to the Received header on the form "polling {label} account {user}", where {label} is the account label (from the specified rcfile, normally ~/.fetchmailrc) and {user} is the username which is used to log on to the mail server. This header can be used to make filtering email where no useful header information is available and you want mail from different accounts sorted into different mailboxes (this could, for example, occur if you have an account on the same server running a mailing list, and are subscribed to the list using that account). The default is not adding any such header. In .fetchmailrc, this is called ’tracepolls’.

RETRIEVAL FAILURE MODES

When forwarding to an MDA, however, there is more possibility of error. Some MDAs are ’safe’ and reliably return a nonzero status on any delivery error, even one due to temporary resource limits. The maildrop(1) program is like this; so are most programs designed as mail transport agents, such as sendmail(1), including the sendmail wrapper of Postfix and exim(1). These programs give back a reliable positive acknowledgement and can be used with the mda option with no risk of mail loss. Unsafe MDAs, though, may return 0 even on delivery failure. If this happens, you will lose mail.

The normal mode of fetchmail is to try to download only ’new’ messages, leaving untouched (and undeleted) messages you have already read directly on the server (or fetched with a previous fetchmail --keep). But you may find that messages you’ve already read on the server are being fetched (and deleted) even when you don’t specify --all. There are several reasons this can happen.

One could be that you’re using POP2. The POP2 protocol includes no representation of ’new’ or ’old’ state in messages, so fetchmail must treat all messages as new all the time. But POP2 is obsolete, so this is unlikely.

A potential POP3 problem might be servers that insert messages in the middle of mailboxes (some VMS implementations of mail are rumored to do this). The fetchmail code assumes that new messages are appended to the end of the mailbox; when this is not true it may treat some old messages as new and vice versa. Using UIDL whilst setting fastuidl 0 might fix this, otherwise, consider switching to IMAP.

Yet another POP3 problem is that if they can’t make tempfiles in the user’s home directory, some POP3 servers will hand back an undocumented response that causes fetchmail to spuriously report "No mail".

The IMAP code uses the presence or absence of the server flag \Seen to decide whether or not a message is new. This isn’t the right thing to do, fetchmail should check the UIDVALIDITY and use UID, but it doesn’t do that yet. Under Unix, it counts on your IMAP server to notice the BSD-style Status flags set by mail user agents and set the \Seen flag from them when appropriate. All Unix IMAP servers we know of do this, though it’s not specified by the IMAP RFCs. If you ever trip over a server that doesn’t, the symptom will be that messages you have already read on your host will look new to the server. In this (unlikely) case, only messages you fetched with fetchmail --keep will be both undeleted and marked old.

In ETRN and ODMR modes, fetchmail does not actually retrieve messages; instead, it asks the server’s SMTP listener to start a queue flush to the client via SMTP. Therefore it sends only undelivered messages.

SPAM FILTERING

Newer versions of sendmail return an error code of 571.

According to RFC2821, the correct thing to return in this situation is 550 "Requested action not taken: mailbox unavailable" (the draft adds "[E.g., mailbox not found, no access, or command rejected for policy reasons].").

Older versions of the exim MTA return 501 "Syntax error in parameters or arguments".

The postfix MTA runs 554 as an antispam response.

Zmailer may reject code with a 500 response (followed by an enhanced status code that contains more information).

Return codes which fetchmail treats as antispam responses and discards the message can be set with the ’antispam’ option. This is one of the only three circumstance under which fetchmail ever discards mail (the others are the 552 and 553 errors described below, and the suppression of multidropped messages with a message-ID already seen).

If fetchmail is fetching from an IMAP server, the antispam response will be detected and the message rejected immediately after the headers have been fetched, without reading the message body. Thus, you won’t pay for downloading spam message bodies.

By default, the list of antispam responses is empty.

If the spambounce global option is on, mail that is spam-blocked triggers an RFC1892/RFC1894 bounce message informing the originator that we do not accept mail from it. See also BUGS.

SMTP/ESMTP ERROR HANDLING

THE RUN CONTROL FILE

To protect the security of your passwords, your ~/.fetchmailrc may not normally have more than 0600 (u=rw,g=,o=) permissions; fetchmail will complain and exit otherwise (this check is suppressed when --version is on).

You may read the .fetchmailrc file as a list of commands to be executed when fetchmail is called with no arguments.

Run Control Syntax

Comments begin with a ’#’ and extend through the end of the line. Otherwise the file consists of a series of server entries or global option statements in a free-format, token-oriented syntax.

There are four kinds of tokens: grammar keywords, numbers (i.e. decimal digit sequences), unquoted strings, and quoted strings. A quoted string is bounded by double quotes and may contain whitespace (and quoted digits are treated as a string). Note that quoted strings will also contain line feed characters if they run across two or more lines, unless you use a backslash to join lines (see below). An unquoted string is any whitespace-delimited token that is neither numeric, string quoted nor contains the special characters ’,’, ’;’, ’:’, or ’=’.

Any amount of whitespace separates tokens in server entries, but is otherwise ignored. You may use backslash escape sequences (\n for LF, \t for HT, \b for BS, \r for CR, \nnn for decimal (where nnn cannot start with a 0), \0ooo for octal, and \xhh for hex) to embed non-printable characters or string delimiters in strings. In quoted strings, a backslash at the very end of a line will cause the backslash itself and the line feed (LF or NL, new line) character to be ignored, so that you can wrap long strings. Without the backslash at the line end, the line feed character would become part of the string.

Warning: while these resemble C-style escape sequences, they are not the same. fetchmail only supports these eight styles. C supports more escape sequences that consist of backslash (\) and a single character, but does not support decimal codes and does not require the leading 0 in octal notation. Example: fetchmail interprets \233 the same as \xE9 (Latin small letter e with acute), where C would interpret \233 as octal 0233 = \x9B (CSI, control sequence introducer).

Each server entry consists of one of the keywords ’poll’ or ’skip’, followed by a server name, followed by server options, followed by any number of user descriptions. Note: the most common cause of syntax errors is mixing up user and server options.

For backward compatibility, the word ’server’ is a synonym for ’poll’.

You can use the noise keywords ’and’, ’with’, ’has’, ’wants’, and ’options’ anywhere in an entry to make it resemble English. They’re ignored, but but can make entries much easier to read at a glance. The punctuation characters ’:’, ’;’ and ’,’ are also ignored.

Poll vs. Skip

Keyword/Option Summary

Here are the legal global options:

Here are the legal server options:

Here are the legal user options:

Remember that all user options must follow all server options.

In the .fetchmailrc file, the ’envelope’ string argument may be preceded by a whitespace-separated number. This number, if specified, is the number of such headers to skip over (that is, an argument of 1 selects the second header of the given type). This is sometime useful for ignoring bogus envelope headers created by an ISP’s local delivery agent or internal forwards (through mail inspection systems, for instance).

Keywords Not Corresponding To Option Switches

The ’folder’ and ’smtphost’ options (unlike their command-line equivalents) can take a space- or comma-separated list of names following them.

All options correspond to the obvious command-line arguments, except the following: ’via’, ’interval’, ’aka’, ’is’, ’to’, ’dns’/’no dns’, ’checkalias’/’no checkalias’, ’password’, ’preconnect’, ’postconnect’, ’localdomains’, ’stripcr’/’no stripcr’, ’forcecr’/’no forcecr’, ’pass8bits’/’no pass8bits’ ’dropstatus/no dropstatus’, ’dropdelivered/no dropdelivered’, ’mimedecode/no mimedecode’, ’no idle’, and ’no envelope’.

The ’via’ option is for if you want to have more than one configuration pointing at the same site. If it is present, the string argument will be taken as the actual DNS name of the mailserver host to query. This will override the argument of poll, which can then simply be a distinct label for the configuration (e.g. what you would give on the command line to explicitly query this host).

The ’interval’ option (which takes a numeric argument) allows you to poll a server less frequently than the basic poll interval. If you say ’interval N’ the server this option is attached to will only be queried every N poll intervals.

Singledrop vs. Multidrop options

The ’is’ or ’to’ keywords associate the following local (client) name(s) (or server-name to client-name mappings separated by =) with the mailserver user name in the entry. If an is/to list has ’*’ as its last name, unrecognized names are simply passed through. Note that until fetchmail version 6.3.4 inclusively, these lists could only contain local parts of user names (fetchmail would only look at the part before the @ sign). fetchmail versions 6.3.5 and newer support full addresses on the left hand side of these mappings, and they take precedence over any ’localdomains’, ’aka’, ’via’ or similar mappings.

A single local name can be used to support redirecting your mail when your username on the client machine is different from your name on the mailserver. When there is only a single local name, mail is forwarded to that local username regardless of the message’s Received, To, Cc, and Bcc headers. In this case, fetchmail never does DNS lookups.

When there is more than one local name (or name mapping), fetchmail looks at the envelope header, if configured, and otherwise at the Received, To, Cc, and Bcc headers of retrieved mail (this is ’multidrop mode’). It looks for addresses with hostname parts that match your poll name or your ’via’, ’aka’ or ’localdomains’ options, and usually also for hostname parts which DNS tells it are aliases of the mailserver. See the discussion of ’dns’, ’checkalias’, ’localdomains’, and ’aka’ for details on how matching addresses are handled.

If fetchmail cannot match any mailserver usernames or localdomain addresses, the mail will be bounced. Normally it will be bounced to the sender, but if the ’bouncemail’ global option is off, the mail will go to the local postmaster instead. (see the ’postmaster’ global option). See also BUGS.

The ’dns’ option (normally on) controls the way addresses from multidrop mailboxes are checked. On, it enables logic to check each host address that does not match an ’aka’ or ’localdomains’ declaration by looking it up with DNS. When a mailserver username is recognized attached to a matching hostname part, its local mapping is added to the list of local recipients.

The ’checkalias’ option (normally off) extends the lookups performed by the ’dns’ keyword in multidrop mode, providing a way to cope with remote MTAs that identify themselves using their canonical name, while they’re polled using an alias. When such a server is polled, checks to extract the envelope address fail, and fetchmail reverts to delivery using the To/Cc/Bcc headers (See below ’Header vs. Envelope addresses’). Specifying this option instructs fetchmail to retrieve all the IP addresses associated with both the poll name and the name used by the remote MTA and to do a comparison of the IP addresses. This comes in handy in situations where the remote server undergoes frequent canonical name changes, that would otherwise require modifications to the rcfile. ’checkalias’ has no effect if ’no dns’ is specified in the rcfile.

The ’aka’ option is for use with multidrop mailboxes. It allows you to pre-declare a list of DNS aliases for a server. This is an optimization hack that allows you to trade space for speed. When fetchmail, while processing a multidrop mailbox, grovels through message headers looking for names of the mailserver, pre-declaring common ones can save it from having to do DNS lookups. Note: the names you give as arguments to ’aka’ are matched as suffixes -- if you specify (say) ’aka netaxs.com’, this will match not just a hostname netaxs.com, but any hostname that ends with ’.netaxs.com’; such as (say) pop3.netaxs.com and mail.netaxs.com.

The ’localdomains’ option allows you to declare a list of domains which fetchmail should consider local. When fetchmail is parsing address lines in multidrop modes, and a trailing segment of a host name matches a declared local domain, that address is passed through to the listener or MDA unaltered (local-name mappings are not applied).

If you are using ’localdomains’, you may also need to specify ’no envelope’, which disables fetchmail’s normal attempt to deduce an envelope address from the Received line or X-Envelope-To header or whatever header has been previously set by ’envelope’. If you set ’no envelope’ in the defaults entry it is possible to undo that in individual entries by using ’envelope <string>’. As a special case, ’envelope "Received"’ restores the default parsing of Received lines.

The password option requires a string argument, which is the password to be used with the entry’s server.

The ’preconnect’ keyword allows you to specify a shell command to be executed just before each time fetchmail establishes a mailserver connection. This may be useful if you are attempting to set up secure POP connections with the aid of ssh(1). If the command returns a nonzero status, the poll of that mailserver will be aborted.

Similarly, the ’postconnect’ keyword similarly allows you to specify a shell command to be executed just after each time a mailserver connection is taken down.

The ’forcecr’ option controls whether lines terminated by LF only are given CRLF termination before forwarding. Strictly speaking RFC821 requires this, but few MTAs enforce the requirement it so this option is normally off (only one such MTA, qmail, is in significant use at time of writing).

The ’stripcr’ option controls whether carriage returns are stripped out of retrieved mail before it is forwarded. It is normally not necessary to set this, because it defaults to ’on’ (CR stripping enabled) when there is an MDA declared but ’off’ (CR stripping disabled) when forwarding is via SMTP. If ’stripcr’ and ’forcecr’ are both on, ’stripcr’ will override.

The ’pass8bits’ option exists to cope with Microsoft mail programs that stupidly slap a "Content-Transfer-Encoding: 7bit" on everything. With this option off (the default) and such a header present, fetchmail declares BODY=7BIT to an ESMTP-capable listener; this causes problems for messages actually using 8-bit ISO or KOI-8 character sets, which will be garbled by having the high bits of all characters stripped. If ’pass8bits’ is on, fetchmail is forced to declare BODY=8BITMIME to any ESMTP-capable listener. If the listener is 8-bit-clean (as all the major ones now are) the right thing will probably result.

The ’dropstatus’ option controls whether nonempty Status and X-Mozilla-Status lines are retained in fetched mail (the default) or discarded. Retaining them allows your MUA to see what messages (if any) were marked seen on the server. On the other hand, it can confuse some new-mail notifiers, which assume that anything with a Status line in it has been seen. (Note: the empty Status lines inserted by some buggy POP servers are unconditionally discarded.)

The ’dropdelivered’ option controls whether Delivered-To headers will be kept in fetched mail (the default) or discarded. These headers are added by Qmail and Postfix mailservers in order to avoid mail loops but may get in your way if you try to "mirror" a mailserver within the same domain. Use with caution.

The ’mimedecode’ option controls whether MIME messages using the quoted-printable encoding are automatically converted into pure 8-bit data. If you are delivering mail to an ESMTP-capable, 8-bit-clean listener (that includes all of the major MTAs like sendmail), then this will automatically convert quoted-printable message headers and data into 8-bit data, making it easier to understand when reading mail. If your e-mail programs know how to deal with MIME messages, then this option is not needed. The mimedecode option is off by default, because doing RFC2047 conversion on headers throws away character-set information and can lead to bad results if the encoding of the headers differs from the body encoding.

The ’idle’ option is intended to be used with IMAP servers supporting the RFC2177 IDLE command extension, but does not strictly require it. If it is enabled, and fetchmail detects that IDLE is supported, an IDLE will be issued at the end of each poll. This will tell the IMAP server to hold the connection open and notify the client when new mail is available. If IDLE is not supported, fetchmail will simulate it by periodically issuing NOOP. If you need to poll a link frequently, IDLE can save bandwidth by eliminating TCP/IP connects and LOGIN/LOGOUT sequences. On the other hand, an IDLE connection will eat almost all of your fetchmail’s time, because it will never drop the connection and allow other polls to occur unless the server times out the IDLE. It also doesn’t work with multiple folders; only the first folder will ever be polled.

The ’properties’ option is an extension mechanism. It takes a string argument, which is ignored by fetchmail itself. The string argument may be used to store configuration information for scripts which require it. In particular, the output of ’--configdump’ option will make properties associated with a user entry readily available to a Python script.

Miscellaneous Run Control Options

Legal protocol identifiers for use with the ’protocol’ keyword are:

    auto (or AUTO) (legacy, to be removed from future release)
    pop2 (or POP2) (legacy, to be removed from future release)
    pop3 (or POP3)
    sdps (or SDPS)
    imap (or IMAP)
    apop (or APOP)
    kpop (or KPOP)

Legal authentication types are ’any’, ’password’, ’kerberos’, ’kerberos_v4’, ’kerberos_v5’ and ’gssapi’, ’cram-md5’, ’otp’, ’msn’ (only for POP3), ’ntlm’, ’ssh’, ’external’ (only IMAP). The ’password’ type specifies authentication by normal transmission of a password (the password may be plain text or subject to protocol-specific encryption as in CRAM-MD5); ’kerberos’ tells fetchmail to try to get a Kerberos ticket at the start of each query instead, and send an arbitrary string as the password; and ’gssapi’ tells fetchmail to use GSSAPI authentication. See the description of the ’auth’ keyword for more.

Specifying ’kpop’ sets POP3 protocol over port 1109 with Kerberos V4 authentication. These defaults may be overridden by later options.

There are some global option statements: ’set logfile’ followed by a string sets the same global specified by --logfile. A command-line --logfile option will override this. Note that --logfile is only effective if fetchmail detaches itself from the terminal. Also, ’set daemon’ sets the poll interval as --daemon does. This can be overridden by a command-line --daemon option; in particular --daemon~0 can be used to force foreground operation. The ’set postmaster’ statement sets the address to which multidrop mail defaults if there are no local matches. Finally, ’set syslog’ sends log messages to syslogd(8).

DEBUGGING FETCHMAIL

Fetchmail crashing

For solving hardware-induced segfaults, find the faulty component and repair or replace it. <http://www.bitwizard.nl/sig11/> may help you with details.

For solving software-induced segfaults, the developers may need a "stack backtrace".

Enabling fetchmail core dumps

1. To get useful backtraces, fetchmail needs to be installed without getting stripped of its compilation symbols. Unfortunately, most binary packages that are installed are stripped, and core files from symbol-stripped programs are worthless. So you may need to recompile fetchmail. On many systems, you can type

        file ‘which fetchmail‘

to find out if fetchmail was symbol-stripped or not. If yours was unstripped, fine, proceed, if it was stripped, you need to recompile the source code first. You do not usually need to install fetchmail in order to debug it.

2. The shell environment that starts fetchmail needs to enable core dumps. The key is the "maximum core (file) size" that can usually be configured with a tool named "limit" or "ulimit". See the documentation for your shell for details. In the popular bash shell, "ulimit -Sc unlimited" will allow the core dump.

3. You need to tell fetchmail, too, to allow core dumps. To do this, run fetchmail with the -d0 -v options. It is often easier to also add --nosyslog -N as well.

Finally, you need to reproduce the crash. You can just start fetchmail from the directory where you compiled it by typing ./fetchmail, so the complete command line will start with ./fetchmail -Nvd0 --nosyslog and perhaps list your other options.

After the crash, run your debugger to obtain the core dump. The debugger will often be GNU GDB, you can then type (adjust paths as necessary) gdb ./fetchmail fetchmail.core and then, after GDB has started up and read all its files, type backtrace full, save the output (copy & paste will do, the backtrace will be read by a human) and then type quit to leave gdb. Note: on some systems, the core files have different names, they might contain a number instead of the program name, or number and name, but it will usually have "core" as part of their name.

INTERACTION WITH RFC 822

        Return-Path:
        Resent-Sender: (ignored if it doesn’t contain an @ or !)
        Sender: (ignored if it doesn’t contain an @ or !)
        Resent-From:
        From:
        Reply-To:
        Apparently-From:

The originating address is used for logging, and to set the MAIL FROM address when forwarding to SMTP. This order is intended to cope gracefully with receiving mailing list messages in multidrop mode. The intent is that if a local address doesn’t exist, the bounce message won’t be returned blindly to the author or to the list itself, but rather to the list manager (which is less annoying).

In multidrop mode, destination headers are processed as follows: First, fetchmail looks for the Received: header (or whichever one is specified by the ’envelope’ option) to determine the local recipient address. If the mail is addressed to more than one recipient, the Received line won’t contain any information regarding recipient addresses.

Then fetchmail looks for the Resent-To:, Resent-Cc:, and Resent-Bcc: lines. If they exist, they should contain the final recipients and have precedence over their To:/Cc:/Bcc: counterparts. If the Resent-* lines don’t exist, the To:, Cc:, Bcc: and Apparently-To: lines are looked for. (The presence of a Resent-To: is taken to imply that the person referred by the To: address has already received the original copy of the mail.)

CONFIGURATION EXAMPLES

Basic format is:

  poll SERVERNAME protocol PROTOCOL username NAME password PASSWORD

Example:

  poll pop.provider.net protocol pop3 username "jsmith" password "secret1"

Or, using some abbreviations:

  poll pop.provider.net proto pop3 user "jsmith" password "secret1"

Multiple servers may be listed:

  poll pop.provider.net proto pop3 user "jsmith" pass "secret1"
  poll other.provider.net proto pop2 user "John.Smith" pass "My^Hat"

Here’s a version of those two with more whitespace and some noise words:

  poll pop.provider.net proto pop3
      user "jsmith", with password secret1, is "jsmith" here;
  poll other.provider.net proto pop2:
      user "John.Smith", with password "My^Hat", is "John.Smith" here;

This version is much easier to read and doesn’t cost significantly more (parsing is done only once, at startup time).

If you need to include whitespace in a parameter string, enclose the string in double quotes. Thus:

  poll mail.provider.net with proto pop3:
        user "jsmith" there has password "u can’t krak this"
                    is jws here and wants mda "/bin/mail"

You may have an initial server description headed by the keyword ’defaults’ instead of ’poll’ followed by a name. Such a record is interpreted as defaults for all queries to use. It may be overwritten by individual server descriptions. So, you could write:

  defaults proto pop3
        user "jsmith"
  poll pop.provider.net
        pass "secret1"
  poll mail.provider.net
        user "jjsmith" there has password "secret2"

It’s possible to specify more than one user per server (this is only likely to be useful when running fetchmail in daemon mode as root). The ’user’ keyword leads off a user description, and every user specification in a multi-user entry must include it. Here’s an example:

  poll pop.provider.net proto pop3 port 3111
        user "jsmith" with pass "secret1" is "smith" here
        user jones with pass "secret2" is "jjones" here keep

This associates the local username ’smith’ with the pop.provider.net username ’jsmith’ and the local username ’jjones’ with the pop.provider.net username ’jones’. Mail for ’jones’ is kept on the server after download.

Here’s what a simple retrieval configuration for a multidrop mailbox looks like:

  poll pop.provider.net:
        user maildrop with pass secret1 to golux ’hurkle’=’happy’ snark here

This says that the mailbox of account ’maildrop’ on the server is a multidrop box, and that messages in it should be parsed for the server user names ’golux’, ’hurkle’, and ’snark’. It further specifies that ’golux’ and ’snark’ have the same name on the client as on the server, but mail for server user ’hurkle’ should be delivered to client user ’happy’.

Note that fetchmail, until version 6.3.4, did NOT allow full user@domain specifications here, these would never match. Fetchmail 6.3.5 and newer support user@domain specifications on the left-hand side of a user mapping.

Here’s an example of another kind of multidrop connection:

  poll pop.provider.net localdomains loonytoons.org toons.org:
        user maildrop with pass secret1 to * here

This also says that the mailbox of account ’maildrop’ on the server is a multidrop box. It tells fetchmail that any address in the loonytoons.org or toons.org domains (including sub-domain addresses like ’[email protected]’) should be passed through to the local SMTP listener without modification. Be careful of mail loops if you do this!

Here’s an example configuration using ssh and the plugin option. The queries are made directly on the stdin and stdout of imapd via ssh. Note that in this setup, IMAP authentication can be skipped.

poll mailhost.net with proto imap:
        plugin "ssh %h /usr/sbin/imapd" auth ssh;
        user esr is esr here

THE USE AND ABUSE OF MULTIDROP MAILBOXES

Also, note that in multidrop mode duplicate mails are suppressed. A piece of mail is considered duplicate if it has the same message-ID as the message immediately preceding and more than one addressee. Such runs of messages may be generated when copies of a message addressed to multiple users are delivered to a multidrop box.

Header vs. Envelope addresses

Sometimes fetchmail can deduce the envelope address. If the mailserver MTA is sendmail and the item of mail had just one recipient, the MTA will have written a ’by/for’ clause that gives the envelope addressee into its Received header. But this doesn’t work reliably for other MTAs, nor if there is more than one recipient. By default, fetchmail looks for envelope addresses in these lines; you can restore this default with -E "Received" or ’envelope Received’.

As a better alternative, some SMTP listeners and/or mail servers insert a header in each message containing a copy of the envelope addresses. This header (when it exists) is often ’X-Original-To’, ’Delivered-To’ or ’X-Envelope-To’. Fetchmail’s assumption about this can be changed with the -E or ’envelope’ option. Note that writing an envelope header of this kind exposes the names of recipients (including blind-copy recipients) to all receivers of the messages, so the upstream must store one copy of the message per recipient to avoid becoming a privacy problem.

Postfix, since version 2.0, writes an X-Original-To: header which contains a copy of the envelope as it was received.

Qmail and Postfix generally write a ’Delivered-To’ header upon delivering the message to the mail spool and use it to avoid mail loops. Qmail virtual domains however will prefix the user name with a string that normally matches the user’s domain. To remove this prefix you can use the -Q or ’qvirtual’ option.

Sometimes, unfortunately, neither of these methods works. That is the point when you should contact your ISP and ask them to provide such an envelope header, and you should not use multidrop in this situation. When they all fail, fetchmail must fall back on the contents of To/Cc headers (Bcc headers are not available - see below) to try to determine recipient addressees -- and these are unreliable. In particular, mailing-list software often ships mail with only the list broadcast address in the To header.

Note that a future version of fetchmail may remove To/Cc parsing!

When fetchmail cannot deduce a recipient address that is local, and the intended recipient address was anyone other than fetchmail’s invoking user, mail will get lost. This is what makes the multidrop feature risky without proper envelope information.

A related problem is that when you blind-copy a mail message, the Bcc information is carried only as envelope address (it’s removed from the headers by the sending mail server, so fetchmail can see it only if there is an X-\Envelope-To header). Thus, blind-copying to someone who gets mail over a fetchmail multidrop link will fail unless the the mailserver host routinely writes X-Envelope-To or an equivalent header into messages in your maildrop.

In conclusion, mailing lists and Bcc’d mail can only work if the server you’re fetching from (1) stores one copy of the message per recipient in your domain and (2) records the envelope information in a special header (X-Original-To, Delivered-To, X-Envelope-To).

Good Ways To Use Multidrop Mailboxes

On your server, you can alias ’fetchmail-friends’ to ’esr’; then, in your .fetchmailrc, declare ’to esr fetchmail-friends here’. Then, when mail including ’fetchmail-friends’ as a local address gets fetched, the list name will be appended to the list of recipients your SMTP listener sees. Therefore it will undergo alias expansion locally. Be sure to include ’esr’ in the local alias expansion of fetchmail-friends, or you’ll never see mail sent only to the list. Also be sure that your listener has the "me-too" option set (sendmail’s -oXm command-line option or OXm declaration) so your name isn’t removed from alias expansions in messages you send.

This trick is not without its problems, however. You’ll begin to see this when a message comes in that is addressed only to a mailing list you do not have declared as a local name. Each such message will feature an ’X-Fetchmail-Warning’ header which is generated because fetchmail cannot find a valid local name in the recipient addresses. Such messages default (as was described above) to being sent to the local user running fetchmail, but the program has no way to know that that’s actually the right thing.

Bad Ways To Abuse Multidrop Mailboxes

If you’re tempted to use fetchmail to retrieve mail for multiple users from a single mail drop via POP or IMAP, think again (and reread the section on header and envelope addresses above). It would be smarter to just let the mail sit in the mailserver’s queue and use fetchmail’s ETRN or ODMR modes to trigger SMTP sends periodically (of course, this means you have to poll more frequently than the mailserver’s expiry period). If you can’t arrange this, try setting up a UUCP feed.

If you absolutely must use multidrop for this purpose, make sure your mailserver writes an envelope-address header that fetchmail can see. Otherwise you will lose mail and it will come back to haunt you.

Speeding Up Multidrop Checking

This is a convenient but also slow method. To speed it up, pre-declare mailserver aliases with ’aka’; these are checked before DNS lookups are done. If you’re certain your aka list contains all DNS aliases of the mailserver (and all MX names pointing at it - note this may change in a future version) you can declare ’no dns’ to suppress DNS lookups entirely and only match against the aka list.

EXIT CODES

The exit codes returned by fetchmail are as follows:

FILES

ENVIRONMENT

If the environment variable FETCHMAILHOME is set to a valid and existing directory name, fetchmail will read $FETCHMAILHOME/fetchmailrc (the dot is missing in this case), $FETCHMAILHOME/.fetchids and $FETCHMAILHOME/.fetchmail.pid rather than from the user’s home directory. The .netrc file is always looked for in the the invoking user’s home directory regardless of FETCHMAILHOME’s setting.

If the HOME_ETC variable is set, fetchmail will read $HOME_ETC/.fetchmailrc instead of ~/.fetchmailrc.

If HOME_ETC and FETCHMAILHOME are set, HOME_ETC will be ignored.

SIGNALS

If fetchmail is running in daemon mode as non-root, use SIGUSR1 to wake it (this is so SIGHUP due to logout can retain the default action of killing it).

Running fetchmail in foreground while a background fetchmail is running will do whichever of these is appropriate to wake it up.

BUGS AND KNOWN PROBLEMS

Fetchmail cannot handle user names that contain blanks after a "@" character, for instance "demonstr@ti on". These are rather uncommon and only hurt when using UID-based --keep setups, so the 6.3.X versions of fetchmail won’t be fixed.

Please check the NEWS file that shipped with fetchmail for more known bugs than those listed here.

The assumptions that the DNS and in particular the checkalias options make are not often sustainable. For instance, it has become uncommon for an MX server to be a POP3 or IMAP server at the same time. Therefore the MX lookups may go away in a future release.

The mda and plugin options interact badly. In order to collect error status from the MDA, fetchmail has to change its normal signal handling so that dead plugin processes don’t get reaped until the end of the poll cycle. This can cause resource starvation if too many zombies accumulate. So either don’t deliver to a MDA using plugins or risk being overrun by an army of undead.

The --interface option does not support IPv6 and it is doubtful if it ever will, since there is no portable way to query interface IPv6 addresses.

The RFC822 address parser used in multidrop mode chokes on some @-addresses that are technically legal but bizarre. Strange uses of quoting and embedded comments are likely to confuse it.

In a message with multiple envelope headers, only the last one processed will be visible to fetchmail.

Use of some of these protocols requires that the program send unencrypted passwords over the TCP/IP connection to the mailserver. This creates a risk that name/password pairs might be snaffled with a packet sniffer or more sophisticated monitoring software. Under Linux and FreeBSD, the --interface option can be used to restrict polling to availability of a specific interface device with a specific local or remote IP address, but snooping is still possible if (a) either host has a network device that can be opened in promiscuous mode, or (b) the intervening network link can be tapped. We recommend the use of ssh(1) tunnelling to not only shroud your passwords but encrypt the entire conversation.

Use of the %F or %T escapes in an mda option could open a security hole, because they pass text manipulable by an attacker to a shell command. Potential shell characters are replaced by ’_’ before execution. The hole is further reduced by the fact that fetchmail temporarily discards any suid privileges it may have while running the MDA. For maximum safety, however, don’t use an mda command containing %F or %T when fetchmail is run from the root account itself.

Fetchmail’s method of sending bounces due to errors or spam-blocking and spam bounces requires that port 25 of localhost be available for sending mail via SMTP.

If you modify a ~/.fetchmailrc while a background instance is running and break the syntax, the background instance will die silently. Unfortunately, it can’t die noisily because we don’t yet know whether syslog should be enabled. On some systems, fetchmail dies quietly even if there is no syntax error; this seems to have something to do with buggy terminal ioctl code in the kernel.

The -f~- option (reading a configuration from stdin) is incompatible with the plugin option.

The ’principal’ option only handles Kerberos IV, not V.

Interactively entered passwords are truncated after 63 characters. If you really need to use a longer password, you will have to use a configuration file.

A backslash as the last character of a configuration file will be flagged as a syntax error rather than ignored.

Send comments, bug reports, gripes, and the like to the fetchmail-devel list <[email protected]>. An HTML FAQ is available at the fetchmail home page; surf to http://fetchmail.berlios.de/ or do a WWW search for pages with ’fetchmail’ in their titles.

AUTHOR

Most of the code is from Eric S. Raymond <[email protected]>. Too many other people to name here have contributed code and patches.

This program is descended from and replaces popclient, by Carl Harris <[email protected]>; the internals have become quite different, but some of its interface design is directly traceable to that ancestral program.

This manual page has been improved by R. Hannes Beinert and H[’e]ctor Garc[’i]a.

SEE ALSO

The fetchmail home page: <http://fetchmail.berlios.de/>

The maildrop home page: <http://www.courier-mta.org/maildrop/>

APPLICABLE STANDARDS

Note that this list is just a collection of references and not a statement as to the actual protocol conformance or requirements in fetchmail.