Create deployment-deploy04 as future secondary/upgrade
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	TheresNoTime
	May 28 2022, 2:10 PM

Description

Per T309413, I am going to spin up a deployment-deploy04 instance and work on getting it configured.

This task is to track the creation and configuration of the instance — once complete, it will require testing and configuration changes in Jenkins before it can be used.

Hostname

deployment-deploy04

Image

debian-10.0-buster

Flavour

g3.cores4.ram8.disk20

Puppet classes

role::beta::deploymentserver
role::deployment_server
role::aptly::server
profile::ci::slave::labs::common

Hiera config

aptly::manage_nginx: false
profile::ci::slave::labs::common::manage_srv: false

Project hiera config notes

The config explicitly sets deploy03 as the role::aptly::client::servername, so this instance won't be in conflict

Considerations

Check the project has sufficient resource quota available
Check that the hostname deployment-deploy04 is unused
Check the deployment.deploy puppet prefix for any conflicts
Check the project hiera config for any conflicts
Do a codesearch of "deployment-deploy03" and "deployment-deploy04" to check for any conflicts

Edited per T309437#7993618

Related Objects

Mentioned In: T309812: [Tracking] Debian 11 package errors
Mentioned Here: T309812: [Tracking] Debian 11 package errors
T309449: Package 'cgroup-bin' has no installation candidate on Debian 11 (modules/mediawiki/manifests/cgroup.pp)
T309659: Two volumes not deleting/creating on deployment-prep
T309413: deployment-deploy03 ran out of memory twice while trying to perform a WikiLambda db migration

Event Timeline

TheresNoTime created this task.May 28 2022, 2:10 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 28 2022, 2:10 PM

Mentioned in SAL (#wikimedia-releng) [2022-05-28T19:08:59Z] <TheresNoTime> deployment-deploy04 live, not referenced by anything T309437

As expected, a bunch of requirements don't work with Bullseye — been slowly working through the ones I can, and will figure out how to correctly propose a patch once ready

(e.g. in modules/mediawiki/manifests/cgroup.pp)

if debian::codename::eq('bullseye') {
    ensure_packages('cgroup-tools')
} else {
    ensure_packages('cgroup-bin')
}

Just to note I am being very cautious (perhaps a little overcautious!), as I don't want to disrupt the beta cluster

Note that 20GB of storage is inadequate for a deploy server. deploy03 has a total of 60GB of storage split between the root filesystem (20GB, ~6GB used) and the /srv filesystem (40GB, ~32GB used). As it stands deploy04 has a full filesystem:

root@deployment-deploy04:~# df -h -t ext4
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        20G   20G     0 100% /

In T309437#7971429, @dancy wrote:
Note that 20GB of storage is inadequate for a deploy server. deploy03 has a total of 60GB of storage split between the root filesystem (20GB, ~6GB used) and the /srv filesystem (40GB, ~32GB used). As it stands deploy04 has a full filesystem:
root@deployment-deploy04:~# df -h -t ext4
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        20G   20G     0 100% /

Ah thanks @dancy — I didn't feel it necessary to attach volumes while still figuring things out, but clearly I needed to.....

Stalled: working through getting incompatibilities resolved (i.e. T309449: Package 'cgroup-bin' has no installation candidate on Debian 11 (modules/mediawiki/manifests/cgroup.pp))

TheresNoTime mentioned this in T309812: [Tracking] Debian 11 package errors.Jun 2 2022, 8:23 PM

What is the goal in building this as bullseye? Shouldn't it aim to match production, which is still buster?

E.g. reviewing the package list in T309812: [Tracking] Debian 11 package errors I don't think we have the bandwith to maintain PHP 7.2/7.4 packages for buster and bullseye.

In T309437#7993618, @Legoktm wrote:

What is the goal in building this as bullseye? Shouldn't it aim to match production, which is still buster?

I admit I didn't word the task overly well — "future" here meaning "a canary for when we move to bullseye"

In T309437#7993618, @Legoktm wrote:

E.g. reviewing the package list in T309812: [Tracking] Debian 11 package errors I don't think we have the bandwith to maintain PHP 7.2/7.4 packages for buster and bullseye.

I agree, but the answer to that is surely "let's stop using something which literally went end-of-life a year and a half ago"...? 🤷‍♀️ ( /lh )

I had a bit more of a think on this, and you're quite right — updated the task & will recreate this as a debian-10.0-buster instance

TheresNoTime changed the task status from Stalled to In Progress.Jun 10 2022, 2:37 PM

This is fairly pointless the more I think about it 🤷‍♀️ will delete the VM and save some quota :)

Create deployment-deploy04 as future secondary/upgradeClosed, DeclinedPublicActions