Mailman3 has multiple components. Each doing a different thing. This is a simplified overview.
- mailman3 is the service that handles emails and talks to MTA (in our case exim4) through LMTP/SMTP
- mailman3-web or postorius is a django-based web service that handles settings, subscriptions, etc. It doesn't do anything on its own (except user management) but it talks internally to the mailman3 service through REST API.
- hyperkitty (being used in the same webservice) is responsible for archives. It handles search, etc.
- Search backend: Is xapian and the search index is in /var/lib/mailman3/web/fulltext_xapian_index
- Databases: Two databases both in m5:
- mailman3 which mailman's core service talks to. It holds memberships, lists, etc.
- mailman3web which postorius and hyperkitty talk to. It holds archive of all emails and is rather big.
- MTA: It's exim4 (see Mailman/MTA for more information)
Disable or re-enable a mailing list
Note: unless you know what you're doing, don't actually do this. It most likely does not do what you want. Just disable the list (see above) instead.
Remove a message from the mailing list archives
Delete the whole thread
- Go to the thread in hyperkitty (example).
- Click on "Delete this thread"
Delete a single message
- Go to the message in hyperkitty (this can be tricky, go to the thread, click on the chain icon aka "Permanant link to this email"). example
- Click on "Delete this message"
Export a listing of all subscribers to a mailing list Check if an email address is subscribed to any lists Remove an individual from all mailing lists
- ssh to lists1001.wikimedia.org
- Run "sudo mailman-wrapper delmembers -m EMAIL_ADDRESS --fromall"
- ssh to lists1001.wikimedia.org.
- Run "sudo mailman-web changepassword USERNAME"
Import from Google Groups
It is straightforward to migrate a list from Google Groups to Mailman. A list owner in Google should use Google Takeout (see documentation
) to export the member list (members.csv) and archives (topics.mbox). File a task requesting a new list and include a link to the Takeout result (if the list contents are private, provide this information to the Mailman admins/SRE team privately).
Copy the mbox to the mailing list server and run:
sudo mailman-web hyperkitty_import -l LISTNAME@lists.wikimedia.org /path/to/topics.mbox
This might take a while because of the "Synchronizing properties with Mailman" step, that's fine.
Then update the search index:
sudo mailman-web update_index_one_list LISTNAME@lists.wikimedia.org
All done! Make sure to have the list owner in Google disable the group there, and remind subscribers to update the lists' email address to LISTNAME@lists.wikimedia.org.
A new list can be added to Gmane very easily if the first message has not already been sent or history is not important. If the history needs to be imported, a ticket should be file in Phabricator with projects 'Wikimedia-Mailing-list' and 'operations' (easy link
stating which list needs to imported and to which Gmane group if it exists already.
Authorized recipients for email@example.com Anyone with a signed NDA can be on the firstname.lastname@example.org mailing list. For a list of signed NDAs for staff/contractors there is T83783
and ldap/nda (or ldap/wmf).
for comments made about past migrations (lily to sodium, sodium to fermium, fermium to lists1001).
When upgrading Mailman3 packages for a version that includes schema changes, use the following process.
- Downtime host in icinga
- Disable puppet
- Manually stop mailman3 and mailman3-web systemd services
- Run apt to install the new packages
- Run sudo mailman-wrapper help to apply the mailman3 schema updates (yes, running just the "help" command will apply the schema changes)
- Run sudo mailman-wrapper help again to make sure the schema update didn't obviously break anything.
- Run sudo mailman-web migrate to apply the mailman3 schema updates
- Clear static CSS/JS caches: sudo mailman-web collectstatic --clear --noinput && sudo mailman-web compress
- Start mailman3 and mailman3-web systemd services again, keep an eye on errors in the logs
- Re-enable puppet and remove icinga downtime if it hasn't expired yet.
The new Mailman3 setup lives on lists1001
. Some state is under /var/lib/mailman3/, the global configuration is in /etc/mailman/ via Puppet.
The mail server used is Exim
, the web server used is Apache
We are currently using custom backported packages
of the mailman3 suite. A few patches have been cherry-picked from upstream to fix issues we've run into. The packaging Git repositories can be found on apt1001 in legoktm's home directory.
There is currently one change
that is hotpatched and not in the package yet.
To get Mailman running with Apache the puppet class role::lists is applied which uses module mailman. Puppet module mailman uses puppet module Apache which sets up Apache and the site config is in the template lists.wikimedia.org.erb.
is backed up to backup1001
. All of /var/lib/mailman3 is backed up, which potentially includes on-disk templates and the search index. We do not back up /var/lib/mailman3/web which comes from the package and digests that haven't been sent yet.
The database lives on m5, which is also regularly backed up into Bacula, too.
Because mail delivery and transport should be reliable, I have tested what happens in certain failure modes, e.g. when SpamAssassin's spamd daemon is not running.
Because of the /defer_ok modifiers in the Exim ACLs, Exim will act as if no spam filtering attempts are made when spamd is not running, and will accept the message. The following lines are logged:
spam acl condition: warning - spamd connection to 127.0.0.1, port 783 failed: Connection refused
spam acl condition: all spamd servers failed
H=xxx.xxxxxxx.xx [xx.xx.xx.xx]:xxxx I=[126.96.36.199]:25 U=exim Warning: ACL "warn" statement skipped: condition test deferred
If the Mailman queue runner daemons are not running, incoming messages will still get delivered to the Mailman queue by Exim. However, nothing else will happen until the Mailman processes are started.
The following people have "superuser" access via the web UI for admin access to all lists:
- Nick Wilson (WMF)
- "Administrator" (all SRE members or users with root)
- "WMFOffice" (Trust and Safety team)
Last edited on 15 September 2021, at 18:57
Content is available under CC BY-SA 3.0
unless otherwise noted.