Page MenuHomePhabricator
Create Task
Maniphest T227432

Replace Varnish backends with ATS on cache text nodes
Closed, ResolvedPublic
Actions

Description

This is the tracking task for the conversion of cache_text on-disk caches from Varnish to ATS. See T226589 for the similar, already completed, conversion of cache_upload.

DCs should be converted starting with the outer-most DCs, for example in the following order:

  • ulsfo
  • eqsin
  • esams
  • codfw
  • eqiad

Details

SubjectRepoBranchLines +/-
ATS: unset client req Accept-Encoding on ats-beoperations/puppetproduction+17 -12
cache: reimage cp1089 as text_atsoperations/puppetproduction+3 -8
Revert "Revert "cache: reimage cp2023 as text_ats""operations/puppetproduction+3 -8
Revert "Revert "cache: reimage cp3064 as text_ats""operations/puppetproduction+3 -8
cache: reimage cp1087 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp1085 as text_atsoperations/puppetproduction+4 -4
ATS: lookup cache for cookie requestsoperations/puppetproduction+8 -5
ATS: use set_server_resp_no_store, do not hide CCoperations/puppetproduction+14 -48
ATS: test setup for default.luaoperations/puppetproduction+5 -4
ATS: improve session/token matchoperations/puppetproduction+15 -7
ATS: mark uncacheable responses as 'pass' in X-Cache-Intoperations/puppetproduction+26 -6
ATS: pass uncacheable requestsoperations/puppetproduction+24 -0
cache: reimage cp1083 as text_atsoperations/puppetproduction+4 -4
Revert "ATS: explicitly skip the cache instead of hiding CC"operations/puppetproduction+41 -7
Revert "ATS: do not coalesce uncacheable requests"operations/puppetproduction+0 -1
ATS: do not coalesce uncacheable requestsoperations/puppetproduction+1 -0
ATS: explicitly skip the cache instead of hiding CCoperations/puppetproduction+6 -40
Revert "cache: reimage cp3064 as text_ats"operations/puppetproduction+8 -3
late_command: remove cpNNNN mkfs stuffoperations/puppetproduction+2 -16
cache: reimage cp1081 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp1079 as text_atsoperations/puppetproduction+4 -4
otrs/phabricator: do not assume text nodes are definedoperations/puppetproduction+2 -6
cache: reimage cp1077 as text_atsoperations/puppetproduction+4 -4
Revert "cache: reimage cp2023 as text_ats"operations/puppetproduction+8 -3
cache: reimage cp2023 as text_atsoperations/puppetproduction+3 -8
cache: reimage cp2019 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2016 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2013 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2012 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2010 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2007 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2006 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp2004 as text_atsoperations/puppetproduction+4 -4
cache_text codfw: read ats-be etcd keysoperations/puppetproduction+1 -0
cache: reimage cp2001 as text_atsoperations/puppetproduction+8 -3
ATS: network settings for ats-beoperations/puppetproduction+6 -0
cache: reimage cp3064 as text_atsoperations/puppetproduction+3 -8
cache: reimage cp3062 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp3060 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp3058 as text_atsoperations/puppetproduction+6 -6
cache: reimage cp3054 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp3052 as text_atsoperations/puppetproduction+4 -4
ATS: move backend::storage_elements settings to profileoperations/puppetproduction+0 -27
ATS: use nvme disk for cp3050 ats-be cacheoperations/puppetproduction+3 -1
cache_text esams: read ats-be etcd keysoperations/puppetproduction+1 -0
cache: reimage cp3050 as text_atsoperations/puppetproduction+8 -3
ATS: remap stream.wmo.org requests on ats-tls as welloperations/puppetproduction+3 -0
ATS: remap stream.wm.org websocket requestsoperations/puppetproduction+3 -1
cumin: aliases: cache::text_ats is a thing nowoperations/puppetproduction+2 -2
cache: reimage cp5012 as text_atsoperations/puppetproduction+3 -8
cache: reimage cp5011 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp5010 as text_atsoperations/puppetproduction+4 -4
varnish: make hitrate dstat plugin work w/o varnish-beoperations/puppetproduction+11 -4
cache: reimage cp5009 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp5008 as text_atsoperations/puppetproduction+4 -4
cache_text eqsin: read ats-be etcd keysoperations/puppetproduction+1 -0
cache: reimage cp5007 as text_atsoperations/puppetproduction+8 -3
prometheus: load text_ats varnish targetsoperations/puppetproduction+2 -1
prometheus: add text_ats mtail targetsoperations/puppetproduction+6 -0
cache: reimage cp4032 as text_atsoperations/puppetproduction+3 -8
cache: reimage cp4031 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp4030 as text_atsoperations/puppetproduction+4 -4
kibana: add discovery recordoperations/dnsmaster+3 -1
kibana: add discovery recordoperations/dnsmaster+2 -0
cache: reimage cp4029 as text_atsoperations/puppetproduction+4 -4
cache: reimage cp4028 as text_atsoperations/puppetproduction+4 -4
cache_text ulsfo: read ats-be etcd keysoperations/puppetproduction+1 -0
cache: reimage cp4027 as text_atsoperations/puppetproduction+8 -3
ATS: include tls profile in cache::text_ats roleoperations/puppetproduction+2 -0
ATS: Vary-slotting for PHP7operations/puppetproduction+59 -0
ATS: cache responses to cookiesoperations/puppetproduction+55 -1
ATS: log Cookie in labs toooperations/puppetproduction+1 -1
ATS: add X-ATS-Timestampoperations/puppetproduction+2 -0
ATS: log Cookiesoperations/puppetproduction+1 -1
ATS: perform MW and RB mangling after cache lookupoperations/puppetproduction+3 -1
Revert "ATS: temporarily use plain HTTP to access docker-registry"operations/puppetproduction+1 -2
ATS: temporarily use plain HTTP to access docker-registryoperations/puppetproduction+2 -1
phabricator::main: whitelist ATS hostsoperations/puppetproduction+2 -1
docker_registry_ha: allow eqiad/codfw varnish/ATS text nodesoperations/puppetproduction+3 -3
ATS: get rid of alternate_domains not overriding cachingoperations/puppetproduction+0 -104
prometheus: fetch cache_text atsmtail@backend metricsoperations/puppetproduction+13 -0
cache_text eqiad: read ats-be etcd keysoperations/puppetproduction+2 -0
cache: ATS storage configuration for cp1075operations/puppetproduction+3 -0
cache: convert cp1075 to text_ats (hiera/conftool)operations/puppetproduction+3 -2
cache: reimage cp1075 as text_atsoperations/puppetproduction+6 -1
ATS: enable compress.so everywhereoperations/puppetproduction+0 -2
ATS: add icinga check for traffic_server restartsoperations/puppetproduction+13 -4
ATS: add icinga check for traffic_server restartsoperations/puppetproduction+12 -4
ATS: enable compress.so for upload@eqsinoperations/puppetproduction+1 -1
Revert "ATS: unset Accept-Encoding"operations/puppetproduction+0 -17
Revert "ATS: leave AE removal to Lua"operations/puppetproduction+9 -0
ATS: compress.so only cache compressed/decompressed variantoperations/puppetproduction+1 -1
Revert "Revert "ATS: enable compress plugin on cp5002""operations/puppetproduction+1 -0
Revert "ATS: enable compress plugin on cp5002"operations/puppetproduction+0 -1
ATS: use proper origin for grafana.wm.orgoperations/puppetproduction+4 -1
ATS: leave AE removal to Luaoperations/puppetproduction+0 -9
ATS: enable compress plugin on cp5002operations/puppetproduction+1 -0
ATS: set minimum-content-length for compress pluginoperations/puppetproduction+14 -0
ATS: unset Accept-Encodingoperations/puppetproduction+17 -0
ATS: disable compress pluginoperations/puppetproduction+1 -1
ATS: add remap rule bugs.wikimedia.org -> phabricatoroperations/puppetproduction+4 -0
ATS: add profile::base::nameserversoperations/puppetproduction+2 -0
ATS: add prometheus::varnishkafka_exporter::configoperations/puppetproduction+11 -0
ATS: add {upload,maps}_domain to text_ats settingsoperations/puppetproduction+2 -0
ATS: unify common trafficserver settingsoperations/puppetproduction+337 -674
ATS: add support for the compress plugin and enable itoperations/puppetproduction+44 -18
ATS: save and restore CC/Expires when forcing no-cacheoperations/puppetproduction+39 -7
ATS: do not cache Authorization responsesoperations/puppetproduction+3 -1
ATS: Vary-slotting for X-Forwarded-Protooperations/puppetproduction+18 -0
ATS: add-vary Lua pluginoperations/puppetproduction+124 -0
ATS: w.wiki rewrite to metaoperations/puppetproduction+48 -3
ATS: gracefully fail request coalescingoperations/puppetproduction+1 -1
ATS: do not cache responses to cookiesoperations/puppetproduction+1 -1
ATS: split the cache for beta variant of the mobile siteoperations/puppetproduction+33 -1
cache: add role::cache::text_atsoperations/puppetproduction+608 -0
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ops-monitoring-bot added a comment.Nov 21 2019, 1:50 PM

Completed auto-reimage of hosts:

['cp1077.eqiad.wmnet']

and were ALL successful.

gerritbot added a comment.Nov 21 2019, 1:51 PM

Change 552245 merged by Ema:
[operations/puppet@production] otrs/phabricator: do not assume text nodes are defined

https://gerrit.wikimedia.org/r/552245

Stashbot added a comment.Nov 21 2019, 1:59 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-21T13:59:50Z] <ema> pool cp1077 with ATS backend T227432

gerritbot added a comment.Nov 21 2019, 2:46 PM

Change 552273 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1079 as text_ats

https://gerrit.wikimedia.org/r/552273

Stashbot added a comment.Nov 21 2019, 2:49 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-21T14:49:32Z] <ema> depool cp1079 and reimage as text_ats T227432

gerritbot added a comment.Nov 21 2019, 2:50 PM

Change 552273 merged by Ema:
[operations/puppet@production] cache: reimage cp1079 as text_ats

https://gerrit.wikimedia.org/r/552273

ops-monitoring-bot added a comment.Nov 21 2019, 2:53 PM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1079.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201911211453_ema_89543.log.

ops-monitoring-bot added a comment.Nov 21 2019, 3:18 PM

Completed auto-reimage of hosts:

['cp1079.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Nov 21 2019, 3:30 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-21T15:30:14Z] <ema> pool cp1079 with ATS backend T227432

gerritbot added a comment.Nov 22 2019, 8:46 AM

Change 552468 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1081 as text_ats

https://gerrit.wikimedia.org/r/552468

Stashbot added a comment.Nov 22 2019, 8:49 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-22T08:49:52Z] <ema> depool cp1081 and reimage as text_ats T227432

gerritbot added a comment.Nov 22 2019, 8:50 AM

Change 552468 merged by Ema:
[operations/puppet@production] cache: reimage cp1081 as text_ats

https://gerrit.wikimedia.org/r/552468

ops-monitoring-bot added a comment.Nov 22 2019, 8:51 AM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1081.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201911220851_ema_38517.log.

ops-monitoring-bot added a comment.Nov 22 2019, 9:17 AM

Completed auto-reimage of hosts:

['cp1081.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Nov 22 2019, 9:28 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-22T09:28:10Z] <ema> pool cp1081 with ATS backend T227432

gerritbot added a comment.Nov 22 2019, 5:34 PM

Change 552547 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] late_command: remove cpNNNN mkfs stuff

https://gerrit.wikimedia.org/r/552547

gerritbot added a comment.Nov 22 2019, 5:36 PM

Change 552547 merged by BBlack:
[operations/puppet@production] late_command: remove cpNNNN mkfs stuff

https://gerrit.wikimedia.org/r/552547

gerritbot added a comment.Nov 25 2019, 2:39 PM

Change 552825 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Revert "cache: reimage cp3064 as text_ats"

https://gerrit.wikimedia.org/r/552825

Stashbot added a comment.Nov 25 2019, 2:45 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-25T14:45:58Z] <ema> depool cp3064 and reimage with varnish-be T227432

gerritbot added a comment.Nov 25 2019, 2:47 PM

Change 552825 merged by Ema:
[operations/puppet@production] Revert "cache: reimage cp3064 as text_ats"

https://gerrit.wikimedia.org/r/552825

Stashbot added a comment.Nov 25 2019, 3:49 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-25T15:49:46Z] <ema> pool cp3064 with varnish-be T227432

gerritbot added a comment.Nov 25 2019, 5:24 PM

Change 552862 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: do not coalesce uncacheable requests

https://gerrit.wikimedia.org/r/552862

• Gilles subscribed.Nov 26 2019, 1:30 PM
This comment was removed by • Gilles.
gerritbot added a comment.Nov 26 2019, 1:59 PM

Change 552076 merged by Ema:
[operations/puppet@production] ATS: explicitly skip the cache instead of hiding CC

https://gerrit.wikimedia.org/r/552076

gerritbot added a comment.Nov 26 2019, 2:03 PM

Change 552862 merged by Ema:
[operations/puppet@production] ATS: do not coalesce uncacheable requests

https://gerrit.wikimedia.org/r/552862

Maintenance_bot removed a project: Patch-For-Review.Nov 26 2019, 2:10 PM
gerritbot added a comment.Nov 26 2019, 2:54 PM

Change 553123 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Revert "ATS: explicitly skip the cache instead of hiding CC"

https://gerrit.wikimedia.org/r/553123

gerritbot added a project: Patch-For-Review.Nov 26 2019, 2:54 PM
gerritbot added a comment.Nov 26 2019, 2:59 PM

Change 553125 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Revert "ATS: do not coalesce uncacheable requests"

https://gerrit.wikimedia.org/r/553125

gerritbot added a comment.Nov 26 2019, 3:01 PM

Change 553125 merged by Ema:
[operations/puppet@production] Revert "ATS: do not coalesce uncacheable requests"

https://gerrit.wikimedia.org/r/553125

gerritbot added a comment.Nov 26 2019, 3:07 PM

Change 553123 merged by Ema:
[operations/puppet@production] Revert "ATS: explicitly skip the cache instead of hiding CC"

https://gerrit.wikimedia.org/r/553123

Maintenance_bot removed a project: Patch-For-Review.Nov 26 2019, 3:10 PM
gerritbot added a comment.Nov 26 2019, 3:43 PM

Change 553132 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: disable coalescing for some uncacheable requests

https://gerrit.wikimedia.org/r/553132

gerritbot added a project: Patch-For-Review.Nov 26 2019, 3:43 PM
• ema closed subtask T237687: ATS doesn't support X-Wikimedia-Debug as Resolved.Nov 29 2019, 2:23 PM
gerritbot added a comment.Dec 3 2019, 9:56 AM

Change 554256 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1083 as text_ats

https://gerrit.wikimedia.org/r/554256

Stashbot added a comment.Dec 3 2019, 10:01 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-03T10:01:22Z] <ema> depool cp1083 and reimage as text_ats T227432

gerritbot added a comment.Dec 3 2019, 10:01 AM

Change 554256 merged by Ema:
[operations/puppet@production] cache: reimage cp1083 as text_ats

https://gerrit.wikimedia.org/r/554256

ops-monitoring-bot added a comment.Dec 3 2019, 10:03 AM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1083.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912031003_ema_133779.log.

ops-monitoring-bot added a comment.Dec 3 2019, 10:30 AM

Completed auto-reimage of hosts:

['cp1083.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Dec 3 2019, 10:37 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-03T10:37:32Z] <ema> pool cp1083 with ATS backend T227432

gerritbot added a comment.Dec 6 2019, 11:08 AM

Change 555396 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: mark uncacheable responses as 'pass' in X-Cache-Int

https://gerrit.wikimedia.org/r/555396

gerritbot added a comment.Dec 6 2019, 1:35 PM

Change 553132 merged by Ema:
[operations/puppet@production] ATS: pass uncacheable requests

https://gerrit.wikimedia.org/r/553132

gerritbot added a comment.Dec 6 2019, 2:01 PM

Change 555396 abandoned by Ema:
ATS: mark uncacheable responses as 'pass' in X-Cache-Int

Reason:
The idea here is wrong. Thanks to hit-for-pass, Varnish can skip cache lookups and coalescing for responses that are known to be uncacheable. In ATS hfp does not exist, hence it would be wrong to label as "pass" things for which we did lookup the cache.

https://gerrit.wikimedia.org/r/555396

Maintenance_bot removed a project: Patch-For-Review.Dec 6 2019, 2:10 PM
• ema mentioned this in T238493: Frontend save timing regression on/after 30 October 2019.Dec 9 2019, 10:20 AM
gerritbot added a comment.Dec 10 2019, 1:29 PM

Change 556185 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: improve session/token match

https://gerrit.wikimedia.org/r/556185

gerritbot added a project: Patch-For-Review.Dec 10 2019, 1:29 PM
gerritbot added a comment.Dec 10 2019, 1:50 PM

Change 556185 merged by Ema:
[operations/puppet@production] ATS: improve session/token match

https://gerrit.wikimedia.org/r/556185

gerritbot added a comment.Dec 10 2019, 2:49 PM

Change 556197 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: test setup for default.lua

https://gerrit.wikimedia.org/r/556197

gerritbot added a comment.Dec 10 2019, 2:53 PM

Change 556197 merged by Ema:
[operations/puppet@production] ATS: test setup for default.lua

https://gerrit.wikimedia.org/r/556197

gerritbot added a comment.Dec 10 2019, 3:09 PM

Change 556201 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: use set_server_resp_no_store, do not hide CC

https://gerrit.wikimedia.org/r/556201

gerritbot added a comment.Dec 10 2019, 4:24 PM

Change 556217 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: lookup cache for cookie requests

https://gerrit.wikimedia.org/r/556217

Stashbot added a comment.Dec 11 2019, 9:25 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-11T09:25:12Z] <ema> cp1075: depool ats-be to test set_server_resp_no_store https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/556201/ T227432

gerritbot added a comment.Dec 11 2019, 9:36 AM

Change 556201 merged by Ema:
[operations/puppet@production] ATS: use set_server_resp_no_store, do not hide CC

https://gerrit.wikimedia.org/r/556201

Stashbot added a comment.Dec 11 2019, 9:44 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-11T09:44:58Z] <ema> cp1075: repool ats-be after successful set_server_resp_no_store test P9849 T227432

Stashbot added a comment.Dec 11 2019, 10:03 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-11T10:03:10Z] <ema> cp-ats: apply set_server_resp_no_store patch https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/556201/ to all hosts T227432

gerritbot added a comment.Dec 11 2019, 11:51 AM

Change 556217 merged by Ema:
[operations/puppet@production] ATS: lookup cache for cookie requests

https://gerrit.wikimedia.org/r/556217

Maintenance_bot removed a project: Patch-For-Review.Dec 11 2019, 12:11 PM
gerritbot added a comment.Dec 19 2019, 7:24 AM

Change 559356 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1085 as text_ats

https://gerrit.wikimedia.org/r/559356

gerritbot added a project: Patch-For-Review.Dec 19 2019, 7:24 AM

Change 559357 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1087 as text_ats

https://gerrit.wikimedia.org/r/559357

gerritbot added a comment.Dec 19 2019, 7:24 AM

Change 559358 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Revert "Revert "cache: reimage cp3064 as text_ats""

https://gerrit.wikimedia.org/r/559358

gerritbot added a comment.Dec 19 2019, 7:24 AM

Change 559359 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Revert "Revert "cache: reimage cp2023 as text_ats""

https://gerrit.wikimedia.org/r/559359

Stashbot added a comment.Dec 19 2019, 7:53 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T07:53:23Z] <ema> depool cp1085 and reimage as text_ats T227432

gerritbot added a comment.Dec 19 2019, 7:54 AM

Change 559356 merged by Ema:
[operations/puppet@production] cache: reimage cp1085 as text_ats

https://gerrit.wikimedia.org/r/559356

ops-monitoring-bot added a comment.Dec 19 2019, 7:55 AM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1085.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912190755_ema_74472.log.

ops-monitoring-bot added a comment.Dec 19 2019, 8:23 AM

Completed auto-reimage of hosts:

['cp1085.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Dec 19 2019, 8:31 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T08:31:18Z] <ema> pool cp1085 with ATS backend T227432

Stashbot added a comment.Dec 19 2019, 8:55 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T08:55:23Z] <ema> depool cp1087 and reimage as text_ats T227432

gerritbot added a comment.Dec 19 2019, 8:58 AM

Change 559357 merged by Ema:
[operations/puppet@production] cache: reimage cp1087 as text_ats

https://gerrit.wikimedia.org/r/559357

ops-monitoring-bot added a comment.Dec 19 2019, 8:59 AM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1087.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912190859_ema_88921.log.

Stashbot added a comment.Dec 19 2019, 9:23 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T09:23:07Z] <ema> depool cp3064 and reimage as text_ats T227432

ops-monitoring-bot added a comment.Dec 19 2019, 9:24 AM

Completed auto-reimage of hosts:

['cp1087.eqiad.wmnet']

and were ALL successful.

gerritbot added a comment.Dec 19 2019, 9:25 AM

Change 559358 merged by Ema:
[operations/puppet@production] Revert "Revert "cache: reimage cp3064 as text_ats""

https://gerrit.wikimedia.org/r/559358

ops-monitoring-bot added a comment.Dec 19 2019, 9:26 AM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp3064.esams.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912190926_ema_95045.log.

Stashbot added a comment.Dec 19 2019, 9:39 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T09:39:19Z] <ema> pool cp1087 with ATS backend T227432

ops-monitoring-bot added a comment.Dec 19 2019, 9:56 AM

Completed auto-reimage of hosts:

['cp3064.esams.wmnet']

Of which those FAILED:

['cp3064.esams.wmnet']
Stashbot added a comment.Dec 19 2019, 10:50 AM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T10:50:42Z] <ema> pool cp3064 with ATS backend T227432

gerritbot added a comment.Dec 19 2019, 10:54 AM

Change 559440 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] cache: reimage cp1089 as text_ats

https://gerrit.wikimedia.org/r/559440

Stashbot added a comment.Dec 19 2019, 12:52 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T12:52:29Z] <ema> depool cp2023 and cp1089 for ATS reimages T227432. Reimaged together because of T238817

gerritbot added a comment.Dec 19 2019, 12:54 PM

Change 559359 merged by Ema:
[operations/puppet@production] Revert "Revert "cache: reimage cp2023 as text_ats""

https://gerrit.wikimedia.org/r/559359

gerritbot added a comment.Dec 19 2019, 12:55 PM

Change 559440 merged by Ema:
[operations/puppet@production] cache: reimage cp1089 as text_ats

https://gerrit.wikimedia.org/r/559440

ops-monitoring-bot added a comment.Dec 19 2019, 12:58 PM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp2023.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912191258_ema_139094.log.

ops-monitoring-bot added a comment.Dec 19 2019, 12:59 PM

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1089.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201912191259_ema_139391.log.

ops-monitoring-bot added a comment.Dec 19 2019, 1:24 PM

Completed auto-reimage of hosts:

['cp1089.eqiad.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Dec 19 2019, 1:25 PM

Completed auto-reimage of hosts:

['cp2023.codfw.wmnet']

and were ALL successful.

Stashbot added a comment.Dec 19 2019, 1:33 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T13:33:03Z] <ema> pool cp2023 with ATS backend T227432

Stashbot added a comment.Dec 19 2019, 1:34 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-19T13:34:01Z] <ema> pool cp1089 with ATS backend T227432

• ema closed this task as Resolved.Dec 19 2019, 2:10 PM
• ema claimed this task.
• ema updated the task description. (Show Details)

cp2023 and cp1089 were the last two hosts running Varnish as backend cache. We now have exclusively ats-be across the fleet!

• ema mentioned this in T241239: Cleanup after varnish-be -> ats-be migration.Dec 20 2019, 1:51 PM
Krinkle mentioned this in T233474: Ensure graphs used by Performance account for Varnish-to-ATS migration.Dec 24 2019, 1:11 AM
gerritbot added a comment.Mar 6 2020, 12:50 PM

Change 577551 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: unset client req Accept-Encoding on ats-be

https://gerrit.wikimedia.org/r/577551

gerritbot added a comment.Mar 9 2020, 9:58 AM

Change 577551 merged by Ema:
[operations/puppet@production] ATS: unset client req Accept-Encoding on ats-be

https://gerrit.wikimedia.org/r/577551

• ema mentioned this in T138546: Backend naming in VCL needs to use fqdn+port.Jul 22 2020, 2:57 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL