Mailman: Difference between revisions
→Components: +exim4 |
→Web server setup: not related at all |
||
Line 97: | Line 97: | ||
To get Mailman running with Apache the puppet class role::lists is applied which uses module mailman. |
To get Mailman running with Apache the puppet class role::lists is applied which uses module mailman. |
||
Puppet module mailman uses puppet module Apache which sets up Apache and the site config is in the template lists.wikimedia.org.erb. |
Puppet module mailman uses puppet module Apache which sets up Apache and the site config is in the template lists.wikimedia.org.erb. |
||
See also http://www.gnu.org/software/mailman/mailman-install/node10.html |
|||
=== Backups === |
=== Backups === |
Revision as of 14:42, 30 April 2021
See also
- meta:Mailing lists
- meta:Mailing lists/Administration
- For mailman2 operations: Mailman/Mailman2
Components
Mailman3 has multiple components. Each doing a different thing. This is a simplified overview.
- mailman3 is the service that handles emails and talks to MTA (in our case exim4) through LMTP/SMTP
- mailman3-web or postorius is a django-based web service that handles settings, subscriptions, etc. It doesn't do anything on its own (except user management) but it talks internally to the mailman3 service through REST API.
- hyperkitty (being used in the same webservice) is responsible for archives. It handles search, etc.
- Search backend: Is xapian and the search index is in /var/lib/mailman3/web/fulltext_xapian_index
- Databases: Two databases both in m5:
- mailman3 which mailman's core service talks to. It holds memberships, lists, etc.
- mailman3web which postorius and hyperkitty talk to. It holds archive of all emails and is rather big.
- MTA: It's exim4 (see Mailman/MTA for more information)
How To
Create a mailing list
- Go to https://lists.wikimedia.org/postorius/lists/new/ (as an administrator)
- or ssh to lists1001.wikimedia.org and run "sudo mailman-wrapper create --owner OWNER"
Disable or re-enable a mailing list
Remove a mailing list
- Go to https://lists.wikimedia.org/postorius/lists/LISTNAME.lists.wikimedia.org/delete (as an administrator)
- or ssh to lists1001.wikimedia.org and run "sudo mailman-wrapper remove LISTNAME"
Remove a message from the mailing list archives
- Delete the whole thread
- Go to the thread in hyperkitty (example).
- Click on "Delete this thread"
- Delete a single message
- Go to the message in hyperkitty (this can be tricky, go to the thread, click on the chain icon aka "Permanant link to this email"). example
- Click on "Delete this message"
Export a listing of all subscribers to a mailing list
- Go to members settings: https://lists.wikimedia.org/postorius/lists/LISTNAME.lists.wikimedia.org/members/member/
- Click on "CSV export"
Check if an email address is subscribed to any lists
Remove an individual from all mailing lists
- ssh to lists1001.wikimedia.org
- Run "sudo mailman-wrapper delmembers -m EMAIL_ADDRESS --fromall"
Reset a user's password
- ssh to lists1001.wikimedia.org.
- Run "sudo mailman-web changepassword USERNAME"
Rename a mailing list
Add a list to Gmane
A new list can be added to Gmane very easily if the first message has not already been sent or history is not important. If the history needs to be imported, a ticket should be file in Phabricator with projects 'Wikimedia-Mailing-list' and 'operations' (easy link stating which list needs to imported and to which Gmane group if it exists already.
Docs and links
Authorized recipients for ops@lists.wikimedia.org
Anyone with a signed NDA can be on the ops@lists.wikimedia.org mailing list. For a list of signed NDAs for staff/contractors there is T83783, WMF-NDA and ldap/nda (or ldap/wmf).
Migrate servers
See Mailman/Migration for comments made about past migrations (lily to sodium, sodium to fermium, fermium to lists1001).
Upgrading Mailman3
When upgrading Mailman3 packages for a version that includes schema changes, use the following process.
- Downtime host in icinga
- Disable puppet
- Manually stop
mailman3
andmailman3-web
systemd services - Run apt to install the new packages
- Run
sudo mailman-wrapper help
to apply the mailman3 schema updates (yes, running just the "help" command will apply the schema changes) - Run
sudo mailman-wrapper help
again to make sure the schema update didn't obviously break anything. - Run
sudo mailman-web migrate
to apply the mailman3 schema updates - Clear static CSS/JS caches:
sudo mailman-web collectstatic --clear --noinput && sudo mailman-web compress
- Start mailman3 and mailman3-web systemd services again, keep an eye on errors in the logs
- Re-enable puppet and remove icinga downtime if it hasn't expired yet.
Configuration details
The new Mailman setup lives on lists1001, and uses the standard Debian package mailman. The mailing list state is under /var/lib/mailman/, the global configuration is in /etc/mailman/.
The mail server used is Exim, the web server used is Apache.
Mailman setup
TODO: Add
Mail server setup
See Mailman/MTA
Web server setup
To get Mailman running with Apache the puppet class role::lists is applied which uses module mailman. Puppet module mailman uses puppet module Apache which sets up Apache and the site config is in the template lists.wikimedia.org.erb.
Backups
lists1001 is backed up to helium using Bacula. The path on the source is /var/lib/mailman. The path on the target is whatever you choose in bconsole.
Tested failure modes
Because mail delivery and transport should be reliable, I have tested what happens in certain failure modes, e.g. when SpamAssassin's spamd daemon is not running.
Spamd not running
Because of the /defer_ok modifiers in the Exim ACLs, Exim will act as if no spam filtering attempts are made when spamd is not running, and will accept the message. The following lines are logged:
spam acl condition: warning - spamd connection to 127.0.0.1, port 783 failed: Connection refused spam acl condition: all spamd servers failed H=xxx.xxxxxxx.xx [xx.xx.xx.xx]:xxxx I=[145.97.39.157]:25 U=exim Warning: ACL "warn" statement skipped: condition test deferred
Mailman not running
If the Mailman queue runner daemons are not running, incoming messages will still get delivered to the Mailman queue by Exim. However, nothing else will happen until the Mailman processes are started.
Spam fighting
Monitoring
Who has the passwords?
The following people have the master (site) password that can be used to login to the admin interface of all lists.
- all members of the operations team or users with root access on the ops bastion host
- Patrick Earley (WMF)
- Joe Sutherland (WMF)
- Nick Wilson (WMF)
The following people have the list creator password that can be used to create new lists but not login to existing lists. This section is outdated as of 2019-03-18
- all members of the operations team or users with root access on the ops bastion host
- James Alexander
- (potentially other CA team members via James)
- User:Odder
- User:Barras
The following people have shell access on the list server. They can run mailman binaries which includes changing list configs and passwords.
- all members of the operations team or users with root access on the ops bastion host