Upgrade deployment-prep deploy* hosts to Buster
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	taavi
	Mar 29 2021, 11:38 AM

Description

According to T265653 deploy* hosts work just fine on Buster, replace deployment-deploy[01-02] hosts with Buster hosts.

Tagging RelEng since your help is needed to make the required changes in Jenkins.

deployment-deploy01 also has some sort of apt repo, not sure how that works and what's needed for that.

TODO: are two hosts needed?

Details

Subject	Repo	Branch	Lines +/-
beta: Drop deployment-deploy01 references from dsh/scap, being decom'ed	operations/puppet	production	+0 -3
hieradata: Swap deployment-prep deploy host	operations/puppet	production	+5 -5
docs: Update deployment-prep deploy host	mediawiki/tools/scap	master	+2 -2
jjb: Update Beta Cluster docs for new deploy server	integration/config	master	+5 -5
beta: remove deployment-deploy02	operations/puppet	production	+0 -3

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T306068 Cloud VPS "deployment-prep" project Stretch deprecation
Resolved	taavi	T278641 Migrate deployment-prep away from Debian Stretch to Buster/Bullseye
Resolved	taavi	T278689 Upgrade deployment-prep deploy* hosts to Buster
Resolved	• Bstorm	T277078 Support Cinder for CI docker workers

Event Timeline

taavi created this task.Mar 29 2021, 11:38 AM

Restricted Application added a project: User-Majavah. · View Herald TranscriptMar 29 2021, 11:38 AM

taavi edited projects, added Release-Engineering-Team; removed Release-Engineering-Team-TODO.Mar 29 2021, 11:39 AM

taavi mentioned this in T278641: Migrate deployment-prep away from Debian Stretch to Buster/Bullseye.Mar 29 2021, 6:40 PM

Mentioned in SAL (#wikimedia-releng) [2021-03-30T13:35:54Z] <Majavah> create and install deployment-deploy03 T278689

Mentioned in SAL (#wikimedia-releng) [2021-03-30T14:50:03Z] <Majavah> cherry pick 675807 675814 and 675815 to deployment-puppetmaster to unblock work on deployment-deploy03 until sre has merged those T278689

deployment-deploy03 has now scap set up. I have not done anything to the aptly repo yet. My understanding is that we can just switch the publish Jenkins jobs to that host, have first scap debs deployed to it, and then switch all clients over.

Next step would be to switch Jenkins jobs to it and change hiera values to make deploy03 the primary deployment server. I don't have enough access in Jenkins to do that.

In T278689#6957023, @Majavah wrote:

deployment-deploy03 has now scap set up. I have not done anything to the aptly repo yet. My understanding is that we can just switch the publish Jenkins jobs to that host, have first scap debs deployed to it, and then switch all clients over.

Next step would be to switch Jenkins jobs to it and change hiera values to make deploy03 the primary deployment server. I don't have enough access in Jenkins to do that.

Everyone in the https://ldap.toolforge.org/group/ciadmin group should be able to help i.e., everyone on Release-Engineering-Team

The steps here will be:

Ensure that deployment-deploy03 is setup and ready
Add deployment-deploy03 as a jenkins node: https://integration.wikimedia.org/ci/computer/new
Apply the jenkins label BetaClusterBastion to deployment-deploy03, remove from deployment-deploy01: https://integration.wikimedia.org/ci/computer/deployment-deploy01/configure and https://integration.wikimedia.org/ci/computer/deployment-deploy03/configure
Test the jobs tied to that label: https://integration.wikimedia.org/ci/label/BetaClusterBastion/ (5: beta-code-update-eqiad, beta-mediawiki-config-update-eqiad, beta-publish-deb, beta-scap-eqiad, beta-update-databases-eqiad -- beta-publish-deb last ran 9 mo. ago so maybe it's not used anymore?

RhinosF1 subscribed.Mar 30 2021, 4:08 PM

In T278689#6957108, @thcipriani wrote:

beta-publish-deb last ran 9 mo. ago so maybe it's not used anymore?

Last failure was 9 months ago, last run was ~2.5h ago as it's run after every commit merged to scap.

This is basically blocked on T277078, as it needs profile::ci::slave::labs::common to not require LVM storage, the new host is on Cinder.

taavi added a subtask: T277078: Support Cinder for CI docker workers.Mar 30 2021, 5:47 PM

hashar mentioned this in T277078: Support Cinder for CI docker workers.Mar 30 2021, 8:11 PM

thcipriani edited projects, added Release-Engineering-Team-TODO; removed Release-Engineering-Team.Apr 6 2021, 4:19 PM

thcipriani moved this task from Should be empty (use Release-Engineering-Team) to Watching/External on the Release-Engineering-Team-TODO board.

thcipriani edited projects, added Release-Engineering-Team (Radar); removed Release-Engineering-Team-TODO.Apr 20 2021, 3:33 AM

thcipriani moved this task from Limbo to Watching/External on the Release-Engineering-Team (Radar) board.Apr 20 2021, 3:34 AM

taavi moved this task from Unsorted to Blocked on others on the User-Majavah board.Apr 23 2021, 2:15 PM

Change 700426 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] beta: remove deployment-deploy02

https://gerrit.wikimedia.org/r/700426

gerritbot added a project: Patch-For-Review.Jun 19 2021, 1:40 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-19T13:44:46Z] <majavah> remove deployment-deploy02 T278689

Change 700426 merged by RLazarus:

[operations/puppet@production] beta: remove deployment-deploy02

https://gerrit.wikimedia.org/r/700426

Maintenance_bot removed a project: Patch-For-Review.Jul 13 2021, 5:10 PM

taavi removed taavi as the assignee of this task.Aug 8 2021, 5:46 PM

taavi removed a project: User-Majavah.

Krinkle closed subtask T277078: Support Cinder for CI docker workers as Resolved.Oct 14 2021, 10:12 PM

hashar removed a project: Continuous-Integration-Config.Oct 26 2021, 7:40 AM

hashar unsubscribed.

Mentioned in SAL (#wikimedia-releng) [2021-11-29T20:59:58Z] <James_F> Switching Jenkins CI config for Beta Cluster host from deployment-deploy01 to deployment-deploy03 for T278689