Mailman

See also

• meta:Mailing lists
• meta:Mailing lists/Administration
• For mailman2 operations: Mailman/Mailman2

Components

Mailman3 has multiple components. Each doing a different thing. This is a simplified overview.

• mailman3 is the service that handles emails and talks to MTA (in our case exim4) through LMTP/SMTP
• mailman3-web or postorius is a django-based web service that handles settings, subscriptions, etc. It doesn't do anything on its own (except user management) but it talks internally to the mailman3 service through REST API.
• hyperkitty (being used in the same webservice) is responsible for archives. It handles search, etc.
• Search backend: Is xapian and the search index is in /var/lib/mailman3/web/fulltext_xapian_index
• Databases: Two databases both in m5:
  • mailman3 which mailman's core service talks to. It holds memberships, lists, etc.
  • mailman3web which postorius and hyperkitty talk to. It holds archive of all emails and is rather big.
  • MTA: It's exim4 (see Mailman/MTA for more information)

How To

Create a mailing list

• Go to https://lists.wikimedia.org/postorius/lists/new/ (as an administrator)
• or ssh to lists1001.wikimedia.org and run "sudo mailman-wrapper create --owner OWNER"

Remove a mailing list

• Go to https://lists.wikimedia.org/postorius/lists/LISTNAME.lists.wikimedia.org/delete (as an administrator)
• or ssh to lists1001.wikimedia.org and run "sudo mailman-wrapper remove LISTNAME"

Remove a message from the mailing list archives

Delete the whole thread

• Go to the thread in hyperkitty (example).
• Click on "Delete this thread"

Delete a single message

• Go to the message in hyperkitty (this can be tricky, go to the thread, click on the chain icon aka "Permanant link to this email"). example
• Click on "Delete this message"

Export a listing of all subscribers to a mailing list

• Go to members settings: https://lists.wikimedia.org/postorius/lists/LISTNAME.lists.wikimedia.org/members/member/
• Click on "CSV export"

Remove an individual from all mailing lists

• ssh to lists1001.wikimedia.org
• Run "sudo mailman-wrapper delmembers -m EMAIL_ADDRESS --fromall"

Reset a user's password

• ssh to lists1001.wikimedia.org.
• Run "sudo mailman-web changepassword USERNAME"

Add a list to Gmane

A new list can be added to Gmane very easily if the first message has not already been sent or history is not important. If the history needs to be imported, a ticket should be file in Phabricator with projects 'Wikimedia-Mailing-list' and 'operations' (easy link stating which list needs to imported and to which Gmane group if it exists already.

Docs and links

• Site Administrator docs on gnu.org

Authorized recipients for ops@lists.wikimedia.org

Anyone with a signed NDA can be on the ops@lists.wikimedia.org mailing list. For a list of signed NDAs for staff/contractors there is T83783, WMF-NDA and ldap/nda (or ldap/wmf).

Migrate servers

See Mailman/Migration for comments made about past migrations (lily to sodium, sodium to fermium, fermium to lists1001).

Upgrading Mailman3

When upgrading Mailman3 packages for a version that includes schema changes, use the following process.

1. Downtime host in icinga
2. Disable puppet
3. Manually stop mailman3 and mailman3-web systemd services
4. Run apt to install the new packages
5. Run sudo mailman-wrapper help to apply the mailman3 schema updates (yes, running just the "help" command will apply the schema changes)
6. Run sudo mailman-wrapper help again to make sure the schema update didn't obviously break anything.
7. Run sudo mailman-web migrate to apply the mailman3 schema updates
8. Clear static CSS/JS caches: sudo mailman-web collectstatic --clear --noinput && sudo mailman-web compress
9. Start mailman3 and mailman3-web systemd services again, keep an eye on errors in the logs
10. Re-enable puppet and remove icinga downtime if it hasn't expired yet.

Configuration details

The new Mailman setup lives on lists1001, and uses the standard Debian package mailman. The mailing list state is under /var/lib/mailman/, the global configuration is in /etc/mailman/.

The mail server used is Exim, the web server used is Apache.

Mailman setup

TODO: Add

Mail server setup

See Mailman/MTA

Web server setup

To get Mailman running with Apache the puppet class role::lists is applied which uses module mailman. Puppet module mailman uses puppet module Apache which sets up Apache and the site config is in the template lists.wikimedia.org.erb.

See also http://www.gnu.org/software/mailman/mailman-install/node10.html

Backups

lists1001 is backed up to helium using Bacula. The path on the source is /var/lib/mailman. The path on the target is whatever you choose in bconsole.

Tested failure modes

Because mail delivery and transport should be reliable, I have tested what happens in certain failure modes, e.g. when SpamAssassin's spamd daemon is not running.

Spamd not running

Because of the /defer_ok modifiers in the Exim ACLs, Exim will act as if no spam filtering attempts are made when spamd is not running, and will accept the message. The following lines are logged:

spam acl condition: warning - spamd connection to 127.0.0.1, port 783 failed: Connection refused
spam acl condition: all spamd servers failed
H=xxx.xxxxxxx.xx [xx.xx.xx.xx]:xxxx I=[145.97.39.157]:25 U=exim Warning: ACL "warn" statement skipped: condition test deferred

Mailman not running

If the Mailman queue runner daemons are not running, incoming messages will still get delivered to the Mailman queue by Exim. However, nothing else will happen until the Mailman processes are started.

Spam fighting

See Mailman/Spam fighting

Monitoring

See Mailman/Monitoring

Who has the passwords?

The following people have the master (site) password that can be used to login to the admin interface of all lists.

• all members of the operations team or users with root access on the ops bastion host
• Patrick Earley (WMF)
• Joe Sutherland (WMF)
• Nick Wilson (WMF)

The following people have the list creator password that can be used to create new lists but not login to existing lists. This section is outdated as of 2019-03-18

• all members of the operations team or users with root access on the ops bastion host
• James Alexander
• (potentially other CA team members via James)
• User:Odder
• User:Barras

The following people have shell access on the list server. They can run mailman binaries which includes changing list configs and passwords.

• all members of the operations team or users with root access on the ops bastion host