Mailman

See also

• meta:Mailing lists
• meta:Mailing lists/Administration

How To

Create a mailing list

There are 2 ways to create a mailing list:

1. Via the web interface at http://lists.wikimedia.org/mailman/create - a list's creator password is needed. The site password works as well.
2. At the shell prompt on lists.wikimedia.org (2012-03: sodium). As root, run newlist.

In both cases, it's not necessary to add e-mail aliases anywhere! When creating a new mailing list, make sure it follows the Naming scheme.

Step-by-step procedure

When you create a new mailing list for someone that could potentially not be experienced with Mailman administration, there's a few things that you need to do:

1. Set yourself to be the initial list administrator, which means that you'll get an email with the list admin password.
   1. This will allow you to adjust settings for the list (if you don't have the site password) and also send the list password in one email to two or more list admins.
2. Set "Auto-generate initial list password" to Yes
3. If relevant, also add any extra "Initial list of supported languages". For example, if you're creating a list for Wikimedia Israel, then add Hebrew as an extra supported language.
4. Enter the List creator's or Site password in the box at the bottom and hit "Create List".
5. Now that you've created the list, login to the admin interface using the password sent to you in an email or the site admin password. Add the list admin email addresses in the relevant field in the admin interface.
6. Also add a "terse phrase" for the list (which will show up in the main list directory at https://lists.wikimedia.org/mailman/listinfo).
7. Add the list to m:Mailing lists/Overview, or tell the requester to.
8. If the list's archives are public, add it to Gmane (this needs to be done before the first message is sent to the list); group name must be under gmane.org.wikimedia, or gmane.science.linguistics.wikipedia for Wikipedia language communities' lists.
9. If the list was requested to be set as private, you need to ensure the archives are private and that the subscription rules are changed to "Confirm and approve".
10. Now you can send an email to the list owners letting them know that the list has been created. Here's a template email that you can use (adjusting as appropriate):

Subject: Your new mailing list: <list name> Body: Hi <names of list owners>, Your new mailing for the <purpose of list> has been created! I've made a few tweaks to the mailing list settings, including adding a terse phrase and making the list publicly archived and listed on the main list directory (https://lists.wikimedia.org/mailman/listinfo). Your mailing list admin password is: <paste password from list created email> You need this password to configure your mailing list. You can configure your mailing list at the following web page: https://lists.wikimedia.org/mailman/admin/<list name> The web page for users of your mailing list is: https://lists.wikimedia.org/mailman/listinfo/<list name> Please list and describe your mailing list at the appropriate section of: https://meta.wikimedia.org/wiki/Mailing_lists/Overview If you ever would like to add more mailing list administrators, login to the admin interface, and add their e-mail addresses to the: "The list administrator email addresses" space. Please also make sure you give the mailing list password to them. Every admin logs in with the same password, and there's no way to send a "password reminder" or get the password back from the system... so don't forget it and make sure they all know what it is if it's changed. Please also remember to subscribe yourself to the mailing list, because you're not automatically subscribed. If you have any questions, please feel free to poke me and I'd be more than happy to help. :-)

Disable a mailing list

Sometimes we might want to disable (aka soft-delete) a list that is no longer used while keeping the archives. This will keep the list settings in place like owners, subscribers and so on. In this case it is sufficient to (on sodium):

# list=LISTNAME
# rt=RT_TICKET
# echo "advertised=0" | config_list -i/dev/stdin $list
# mv /var/lib/mailman/lists/{$list,$list.disabled.$rt}
# rm /var/lib/mailman/data/heldmsg-$list-*.pck

Restore a mailing list

Undoing the process above involves the same steps but reversed:

# mv /var/lib/mailman/lists/{$list.disabled.rt_number,$list}
# echo "advertised=1" | config_list -i/dev/stdin $list

Remove a mailing list

To remove a mailing list, at the shell prompt on sodium run:

# rmlist listname

To also remove all archives, use:

# rmlist -a listname

Remove a message from the mailing list archives

Sometimes it's necessary to remove a message from mailing list archive, e.g. when someone's complaining about public visibility in search engines. However, mailing list archives, though public, are no longer indexed by search engines as they are excluded in robots.txt.

Export a listing of all subscribers to a mailing list

• Login to the mailing list server

  /var/lib/mailman/bin/list_members -f -o <file to write to> <list name>

Remove an individual from all mailing lists

Occasionally we need to remove an individual from every mailing list we have, such as when an email address no longer works but we don't want mailman to turn it off due to bounce detection. The remove_members command is the solution - this is a command line utility to remove one or more email addresses from a specific list or from all lists.

• remove an individual from a specific list

 /var/lib/mailman/bin/remove_members mylist user@example.com

• remove two addresses from all lists

 /var/lib/mailman/bin/remove_members --nouserack --fromall user1@example.com user2@example.com

Rename a mailing list

1. read "Q. How do I rename a list?" in mailman FAQ and ask yourself (or others) if this is really necessary. If you still want this:
2. create a new list
   1. use the mailman master password as creator's (authentication) password
   2. follow the standardized naming scheme where possible.
   3. use your email as the initial list creator and receive auto-created password
   4. log in on admin web ui (https://lists.wikimedia.org/mailman/admin/<listname>)
3. before the next step be prepared to change the "real_name" value of the list in the web ui, but don't send it yet. have the mailman master pass ready.
4. go to sodium shell and copy the config.pck (this includes all settings, users, passwords!) and others (pending.pck, request.pck insert pathes, files) from old to new list
5. reload the admindb link and be logged out, because you have just overwritten the users and passwords as well, use the mailman master password and login again
6. immediately change the real_name in web ui (do this quick because it is untested what happens if two lists with the same exist and a mail arrives just then)
7. you should now see other users as list admins and members, and now you can take your time and adjust other settings like the description field and "Prefix for subject line of list postings" and update the name there as well or let others do it
8. copy the archive mbox from old to new .mbox directory and rename it to reflect the new list name.
9. fix permissions (chown list:list *.mbox, chmod 664 *.mbox)
10. use arch to recreate all html files from mbox to fix archive links
11. decide if you want to keep old archives in place, you probably do. dont break URLs.
12. add the old list email address to "acceptable aliases" on the new list web ui (insert details)
13. merge a mail alias to redirect mail to the old list
14. merge an url redirect for the old listinfo page
15. merge on palladium, run puppet on sodium, reload lighttpd (insert details)
16. test url redirect
17. dont forget to subscribe yourself to the new list
18. test mail by announcing the change to the old list address

Docs and links

• Site Administrator docs on gnu.org

Upgrade Mailman

The Mailman package insists on the Mailman queue being empty during the upgrade. As messages are constantly coming in, that's not easy, and stopping the Mailman process doesn't prevent Exim from delivering messages to Mailman either. Therefore, the best solution is to make Exim hold messages on the queue by putting the line

hold_domains = lists.wikimedia.org

in /etc/exim4/exim4.conf, in the main configuration section. Run

# /etc/init.d/exim4 reload

Afterwards. Then, perform the apt-get upgrade, and reverse the Exim4 config.

Alter arbcom-l archive access list

After a previous data leak, the English Wikipedia Arbitration Committee requested a high level of security for access to arbcom-l archives. We now have a second layer of password authentication in addition to Mailman's controls, implemented in lighttpd. Using HTTP authentication allows each user's page views to be tracked in the lighttpd access logs.

To add a user:

• Have the user generate a GPG private key, and have them establish a link between their public key and their Wikipedia user account by posting the public key on Wikipedia. For Windows users, this can be done by following this tutorial.
• Generate a password for them, for example using tr -cd [:alnum:] < /dev/urandom | head -c10
• Choose a username without spaces (Lighttpd bug #2411)
• Update the htdigest file in the puppet private repo using htdigest ~/private/files/lighttpd/htdigest 'arbcom-l archive' <username>
• Push the update out to sodium in the usual way.
• Send the new password to the user using encrypted email.

To do other things:

• Edit palladium:~/private/files/lighttpd/htdigest and push the file out to sodium.

Configuration details

The new Mailman setup lives on sodium, and uses the standard Ubuntu package mailman. The mailing list state is under /var/lib/mailman/, the global configuration is in /etc/mailman/.

The mail server used is Exim, the web server used is lighttpd.

Mailman setup

Mailman has fairly reasonable default values, and doesn't need a lot of changes from the defaults. The following settings were modified in /etc/mailman/mm_cfg.py:

# If you change these, you have to configure your http server
# accordingly (Alias and ScriptAlias directives in most httpds)
DEFAULT_URL_PATTERN = 'http://%s/mailman/'
PRIVATE_ARCHIVE_URL = '/mailman/private'

# Default domain for email addresses of newly created MLs
DEFAULT_EMAIL_HOST = 'lists.wikimedia.org'
# Default host for web interface of newly created MLs
DEFAULT_URL_HOST   = 'lists.wikimedia.org'

Exim recognizes which lists exist under @lists.wikimedia.org, so aliases are only needed in other domains:

# Uncomment this if you configured your MTA such that it
# automatically recognizes newly created lists.
# (see /usr/share/doc/mailman/README.{EXIM,...})
# MTA=None   # Misnomer, suppresses alias output on newlist
MTA=None

# Set Reply-To to the list by default
DEFAULT_REPLY_GOES_TO_LIST = 0

htDig integration

See Mailman-htdig for details. htDig has been integrated in the Wikimedia Mailman Ubuntu package, and just needs to be enabled in mm_cfg.py.

Due to performance problems we were not using/updating the htDig indexes anyway - htDig integration has therefore been removed during the distribution upgrade to Hardy.

Mail server setup

Near the top of the exim4.conf file, there are several macros related to Mailman. These define system-specific settings/locations used by the router(s) and transport(s) in the rest of the configuration file. For a Debian/Ubuntu Mailman package, the following macro's are accurate:

# Mailman
MAILMAN_HOME = /usr/lib/mailman
MAILMAN_LISTS_HOME = /var/lib/mailman
MAILMAN_WRAP = MAILMAN_HOME/mail/mailman
MAILMAN_UID = list
MAILMAN_GID = list

There's a domain list that contains a list of all domains that can "contain" mailing lists, i.e. the domains for which the Mailman router(s) should run. This list is also used as part of the "local domains" list, the list for which this mail server accepts mail and handles it locally.

domainlist mailman_domains = lists.wikimedia.org
domainlist local_domains = +system_domains : +mailman_domains

Main configuration

Several tweaks have been made to the main configuration to make Mailman delivery go smooth.

In case of high load / lots of incoming connections, mail from the local host (including Mailman) and other Wikimedia servers are given preference:

smtp_reserve_hosts = <; 127.0.0.1 ; ::1 ; +wikimedia_nets

For big mailing lists, Mailman needs to send a lot of recipients per mail / connection. Per default, Exim only queues mails that have > 10 recipients, to be delivered by a subsequent queue runner, which can cause significant delays. The default Mailman limit is 500 recipients per connection, so make Exim accept that:

smtp_accept_queue_per_connection = 500

Allow Exim to do 50 deliveries to remote hosts in parallel (this means 50 processes):

remote_max_parallel = 50

Routers

In Exim, the routers determine if a certain e-mail address is accepted for delivery or mail transport, and how it's going to be handled (routed). For Mailman, the following list router accepts a recipient that:

• has a domain in the domain list mailman_domains
• has a Mailman configuration file matching the local part (i.e. the mailing list exists)

Certain postfixes of the localpart, e.g. "-bounces" are accepted as well.

When the router accepts the recipient address, it's set up for delivery using the list transport (see below).

# Mailman list handling. Test the mailing list address without suffix
# first, as a mailing list like wikifi-admin is a valid list name.

list:
        driver = accept
        domains = +mailman_domains
        require_files = MAILMAN_LISTS_HOME/lists/$local_part/config.pck
        transport = list

list_suffix:
        driver = accept
        domains = +mailman_domains
        require_files = MAILMAN_LISTS_HOME/lists/$local_part/config.pck
        local_part_suffix = -bounces : -bounces+* : \
                                -confirm+* : -join : -leave : \
                                -owner : -request : -admin : \
                                -subscribe : -unsubscribe
        transport = list

If the conditions for this router fail (i.e. the router is not run) then the no_more makes sure that no subsequent routers will be tried (in the current configuration there are none that might accept), and the recipient address is failed.

Transports

An Exim transport configures a way of transporting a message, e.g. over the network (SMTP), to a file (MBOX/Maildir/etc) or using a pipe to a process. The following transport sets up delivery to Mailman:

# Mailman pipe transport

list:
        driver = pipe
        command = MAILMAN_WRAP \
                '${if def:local_part_suffix \
                        {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
                        {post}}' \
                $local_part
        current_directory = MAILMAN_LISTS_HOME
        home_directory = MAILMAN_LISTS_HOME
        user = MAILMAN_UID
        group = MAILMAN_GID

For content scanning, temporary mbox files are written to /var/spool/exim4/scan, and deleted after scanning. Similarly, Exim keeps "hints" databases in /var/spool/exim4/db, which are non-essential caches. To improve performance somewhat, these directory is mounted as a tmpfs filesystem, using the following line in /etc/fstab:

tmpfs   /var/spool/exim4/scan   tmpfs   defaults        0       0
tmpfs   /var/spool/exim4/db     tmpfs   defaults        0       0

Mailing list privacy protection

It has happened in the past that by hitting on the Reply All button in one's e-mail client, private info from an internal list leaked to a public mailing list because it was listed in the CC list, and the imprudent sender did not notice. In order to try to catch these incidents, a little filter has been implemented.

1. If the To: or CC: of a message body matches an item in a list of private mailing list addresses, and
2. the list of recipients as known by the mailing list server contains a Wikimedia mailing list that's not a private mailing lists, then
3. the message is bounced with the message Message rejected for privacy protection: The list of recipients contains both private and public lists.

It's possible to circumvent this restriction by sending to the private list as a BCC, Blind Carbon Copy.

This filter is implemented using an Exim system filter:

# Exim filter # Mailing list privacy protection if foranyaddress $h_To:,$h_Cc: ( $thisaddress matches "\\N^(internal-l|private-l)@(lists\.|mail\.)?wiki[mp]edia\.org$\\N" ) then if foranyaddress $recipients ( $thisaddress matches "\\N@lists\.wikimedia\.org$\\N" and $thisaddress does not match "\\N^(internal-l|private-l)@\\N" ) then fail text "Message rejected for privacy protection: The list of recipients contains both private and public mailing lists" endif endif

This filter is enabled in the configuration file using

system_filter = CONFDIR/system_filter

Address header rewriting

It turned out that, after the migration, many users kept sending mails to both the old and the new mailing list addresses, thereby causing duplicate messages. To reduce this, Exim has been configured to rewrite the old mailing list addresses to the new ones in the To: and CC: headers, using the following option on the list transport:

list: ... # Rewrite body headers of old mailing list addresses to new ones headers_rewrite = \N^.*@(mail\.)?wiki[mp]edia\.org$\N "${if exists{MAILMAN_LISTS_HOME/lists/$local_part/config.pck}{$local_part@lists.wikimedia.org}fail}" ct

Web server setup

To get Mailman running with lighttpd, a couple of small changes had to be made to the default configuration file. mod_cgi and mod_redirect need to be loaded:

server.modules              = (
           "mod_access",
           "mod_alias",
           "mod_accesslog",
           "mod_redirect",
           "mod_cgi",
)

To make path /mailman/ invoke the correct CGI scripts, use:

# Mailman
alias.url = (
       "/mailman/"     => "/usr/lib/cgi-bin/mailman/",
       "/pipermail/"   => "/var/lib/mailman/archives/public/",
       "/images/"      => "/usr/share/images/",
)

url.redirect = (
       "^/(index\.html?)?$"    => "http://meta.wikimedia.org/wiki/Mailing_lists/overview",
       "^/mailman/?$"          => "/mailman/listinfo"
)

$HTTP["url"] =~ "^/mailman/" {
       cgi.assign = ( "" => "" )
}

See also http://www.gnu.org/software/mailman/mailman-install/node10.html

SpamAssassin

SpamAssassin is installed using the default Ubuntu spamassassin package. A couple of configuration changes were made.

By default, spamd, if enabled, runs as root. To change this:

# adduser --system --home /var/lock/spamassassin --group --disabled-password --disabled-login spamd

The following settings were modified in /etc/default/spamassassin:

# Change to one to enable spamd
ENABLED=1

User preferences are disabled, spamd listens on the loopback interface only, and runs as user/group spamd:

OPTIONS="--max-children 5 --nouser-config --listen-ip=127.0.0.1 -u spamd -g spamd"

Run spamd with nice level 10:

# Set nice level of spamd
NICE="--nicelevel 10"

Fighting spam in mailman

All list mail is sent through spamassassin which adds headers with a spam status and score.

This does not mean though that it automatically discards any messages. Only messages with a really high spam socre (>= 12) are discarded immediately, below that messages are just tagged with a header. What actions to take based on this information is left to the list admins.

The mailman UI supports this via the configuration variable header_filter_rules aka. 'Spam Filter Regexp' (description: Filter rules to match against the headers of a message.).

This can be found in the administrative interface in Privacy options...-> [Spam filters] -> Spam Filter Regexp.

You can just rely on spam-status:

x-spam-status: yes  

or you can filter by score, example regex from mailman docs:

X-Spam-Score: [*]{3,5}

This line will match from 3 to 5 stars in the value of this field.

another example could be:

X-Spam-Score: [+]{5,}

if the X-Spam-Score is indicated with pluses (+) insted of stars (*), this line matches when 5 or more pluses are encountered on the same line.

Additionally you have to pick an action for that which can be one of:

Defer  Hold  Reject  Discard  Accept

If you use "Hold" you will have to manually check the moderation queue every once in a while, but can avoid possible false positives, if you use "Discard" you don't have to do anything but _might_ discard some ham without noticing, though that is unlikely.

Backups

sodium is backed up to tridge using Amanda. The path on the source is /var/lib/mailman. The path on the target is /data/amanda/...

Tested failure modes

Because mail delivery and transport should be reliable, I have tested what happens in certain failure modes, e.g. when SpamAssassin's spamd daemon is not running.

Spamd not running

Because of the /defer_ok modifiers in the Exim ACLs, Exim will act as if no spam filtering attempts are made when spamd is not running, and will accept the message. The following lines are logged:

spam acl condition: warning - spamd connection to 127.0.0.1, port 783 failed: Connection refused
spam acl condition: all spamd servers failed
H=xxx.xxxxxxx.xx [xx.xx.xx.xx]:xxxx I=[145.97.39.157]:25 U=exim Warning: ACL "warn" statement skipped: condition test deferred

Mailman not running

If the Mailman queue runner daemons are not running, incoming messages will still get delivered to the Mailman queue by Exim. However, nothing else will happen until the Mailman processes are started.

TODO

• Mail server configuration fine tuning
• Mail server configuration documentation
• Mailman configuration fine tuning
• Spam filtering (current config?)
• htdig
• Backup MX
• Automatic mailing list index script (also, 404 handlers, robots.txt...)
• Migrating existing mailing lists, with announcements
• Redirection of old URL to new
• DNS Resolver
• Search engine for archive messages (Including private messages)
• Monitoring
• Backups

Migration

Configuration files can be copied to sodium just fine. Variables that may need to be changed are:

• reply_to_address
• host_name

Most of these can probably be done automatically. Not present in the dumped list configuration file is the list's URLs. A fix_url withlist script is provided to change this.

Archives can be copied by just transferring the .mbox file, and then rebuilding the archive from scratch with arch --wipe.

Migration to sodium

• Files in dirs lists, archives and data should be rsynced again, with Mailman disabled on both sides. dpkg-reconfigure mailman should be rerun.