Log In
CDanis
CDanis (Chris Danis)
SRE @ WMF
Projects (10)
conftool
Component
Infrastructure-Foundations
Group
netops
Component
Security
Tag
SRE-OnFire
Group
Calendar
Today
Clear sailing ahead.
Tomorrow
Clear sailing ahead.
Tuesday
Clear sailing ahead.
User Details
User Since
Nov 5 2018, 2:54 PM (153 w, 6 d)
Availability
Available
IRC Nick
cdanis
LDAP User
CDanis
MediaWiki User
CDanis (WMF) [ Global Accounts ]
Recent Activity
Yesterday
CDanis added a comment to T293536: MediaWiki should support setting a read query time limit.
In T293536#7433184, @Legoktm wrote:
For known slow queries, this would be useful in addition to the automated query killer to limit runtime to <= 60 seconds.
This could also work in the opposite direction, have a global limit that's low, but increase it for known slow queries.
Sat, Oct 16, 12:02 AM · Platform Engineering, Performance-Team, Wikimedia-Rdbms, Sustainability (Incident Followup)
Fri, Oct 15
CDanis added a comment to T292284: Migrate Zuul alerting to Grafana / AlertManager.
BTW, I think this fired today in the releng IRC channel?
Fri, Oct 15, 6:07 PM · observability, Release-Engineering-Team (Doing), Zuul, Continuous-Integration-Infrastructure
CDanis added a subtask for T202061: Implement an accurate and easy to understand status page for all wikis: Unknown Object (Task).
Fri, Oct 15, 4:53 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
CDanis changed the status of T293455: Requesting access to analytics-privatedata-users for echetty from Open to Stalled.
Fri, Oct 15, 3:32 PM · SRE, SRE-Access-Requests
CDanis closed T293241: Requesting access to analytics-privatedata-users for lbowmaker as Resolved.
Access granted to the wmf group in LDAP, so you should be able to access web-based tools.
Fri, Oct 15, 3:28 PM · SRE, SRE-Access-Requests
CDanis added a comment to T293455: Requesting access to analytics-privatedata-users for echetty.
also @DAbad please confirm you approve as well
Fri, Oct 15, 2:54 PM · SRE, SRE-Access-Requests
CDanis updated subscribers of T293455: Requesting access to analytics-privatedata-users for echetty.
@Ottomata or @odimitrijevic please approve, thanks!
Fri, Oct 15, 2:54 PM · SRE, SRE-Access-Requests
CDanis added a comment to T293241: Requesting access to analytics-privatedata-users for lbowmaker.
@lbowmaker Great, thanks. Please paste somewhere in this task the public part of the key (a line of text that will start with something like ssh-ed25519 or ssh-rsa) and I'll get you set up.
Fri, Oct 15, 1:29 PM · SRE, SRE-Access-Requests
Thu, Oct 14
CDanis added a comment to T293253: Requesting access to analytics-privatedata-users for DAbad.
Thanks Andrew! @DAbad if you want shell access, please generate and paste a public key: https://wikitech.wikimedia.org/wiki/SRE/Production_access#Generating_your_SSH_key
Thu, Oct 14, 8:48 PM · SRE, SRE-Access-Requests
CDanis updated subscribers of T293241: Requesting access to analytics-privatedata-users for lbowmaker.
@Ottomata or @odimitrijevic can you please approve? Thanks!
Thu, Oct 14, 8:38 PM · SRE, SRE-Access-Requests
CDanis updated subscribers of T293253: Requesting access to analytics-privatedata-users for DAbad.
@Ottomata or @odimitrijevic can you please approve? Thanks!
Thu, Oct 14, 8:38 PM · SRE, SRE-Access-Requests
CDanis closed T293053: Grant Access to <WMF Group> for <Jacquelinechen> as Resolved.
Access granted!
Thu, Oct 14, 8:35 PM · SRE, LDAP-Access-Requests
CDanis claimed T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Thu, Oct 14, 8:11 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis updated the task description for T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Thu, Oct 14, 8:11 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
Wed, Oct 13
CDanis closed T292955: Requesting access to Analytic Cluster for Muniza as Resolved.
Shell account created and it will be live in ~30 minutes.
Wed, Oct 13, 6:21 PM · SRE, SRE-Access-Requests
CDanis added a comment to T264021: ~1 request/minute to intake-logging.wikimedia.org times out at the traffic/service interface.
Just a quick update that, after a year, this is still happening, on both intake-analytics (50x/minute =~ 0.8x/sec) and intake-logging (1-2x/minute).
Wed, Oct 13, 5:58 PM · Analytics-Kanban, Analytics, SRE
CDanis updated the task description for T292955: Requesting access to Analytic Cluster for Muniza.
Wed, Oct 13, 4:13 PM · SRE, SRE-Access-Requests
CDanis added a comment to T292955: Requesting access to Analytic Cluster for Muniza.
@MunizaA can you confirm that this wikitech user is you? https://ldap.toolforge.org/user/mnz
Wed, Oct 13, 1:49 PM · SRE, SRE-Access-Requests
Tue, Oct 12
CDanis added a comment to T292870: externally-hosted NEL report forwarders for more timely report reception.
In T292870#7415867, @ayounsi wrote:
I'd wary of the complexity of the setup.
Tue, Oct 12, 9:40 PM · Traffic-Icebox, Infrastructure-Foundations, netops, SRE
CDanis reassigned T292955: Requesting access to Analytic Cluster for Muniza from Ottomata to MunizaA.
@MunizaA can you please sign the L3 agreement by clicking on L3 ? thanks!
Tue, Oct 12, 9:22 PM · SRE, SRE-Access-Requests
CDanis closed T292992: Requesting access to analytics-privatedata-users for KCVelaga (WMF) as Resolved.
Access granted!
Tue, Oct 12, 9:20 PM · SRE, SRE-Access-Requests
CDanis added a comment to T291508: Requesting access to analytics-privatedata-access for NRodriguez .
Hi Natalia, just a friendly reminder that this ticket is still awaiting an SSH public key from you.
Tue, Oct 12, 8:19 PM · SRE, SRE-Access-Requests
CDanis closed T292782: Requesting access to analytics-privatedata-users for Clare Ming as Resolved.
Access granted!
Tue, Oct 12, 8:16 PM · SRE, SRE-Access-Requests
CDanis closed T292841: Request access to private data group for Ldelench as Resolved.
Access granted!
Tue, Oct 12, 8:06 PM · SRE, Campaign-Tools, SRE-Access-Requests
CDanis updated the task description for T292841: Request access to private data group for Ldelench.
Tue, Oct 12, 7:53 PM · SRE, Campaign-Tools, SRE-Access-Requests
CDanis assigned T292955: Requesting access to Analytic Cluster for Muniza to Ottomata.
Andrew, can you approve? Thanks!
Tue, Oct 12, 7:52 PM · SRE, SRE-Access-Requests
CDanis assigned T292992: Requesting access to analytics-privatedata-users for KCVelaga (WMF) to Ottomata.
Andrew, can you approve? Thanks!
Tue, Oct 12, 7:52 PM · SRE, SRE-Access-Requests
Fri, Oct 8
CDanis updated the task description for T292870: externally-hosted NEL report forwarders for more timely report reception.
Fri, Oct 8, 7:24 PM · Traffic-Icebox, Infrastructure-Foundations, netops, SRE
CDanis added a subtask for T257527: automatically collect network error reports from users' browsers (Network Error Logging API): T292870: externally-hosted NEL report forwarders for more timely report reception.
Fri, Oct 8, 7:11 PM · Patch-For-Review, Product-Data-Infrastructure, SRE, Goal, Epic
CDanis added a parent task for T292870: externally-hosted NEL report forwarders for more timely report reception: T257527: automatically collect network error reports from users' browsers (Network Error Logging API).
Fri, Oct 8, 7:11 PM · Traffic-Icebox, Infrastructure-Foundations, netops, SRE
CDanis triaged T292870: externally-hosted NEL report forwarders for more timely report reception as Low priority.
Fri, Oct 8, 7:11 PM · Traffic-Icebox, Infrastructure-Foundations, netops, SRE
CDanis updated the title for P17330 Example SLO calculation in PromQL: hypothetical Mediawiki average latency SLO from untitled to Example SLO calculation in PromQL: hypothetical Mediawiki average latency SLO.
Fri, Oct 8, 3:18 PM
Thu, Oct 7
CDanis merged task T292776: Wikipedia not accessible in Russia on 2021-10-07 16:00-17:00UTC into T292792: 2021-10-07 network provider issues causing all Wikimedia sites to be unreachable for many users.
Thu, Oct 7, 9:02 PM · Traffic, SRE
CDanis merged T292776: Wikipedia not accessible in Russia on 2021-10-07 16:00-17:00UTC into T292792: 2021-10-07 network provider issues causing all Wikimedia sites to be unreachable for many users.
Thu, Oct 7, 9:01 PM · Wikimedia-Incident, SRE
CDanis created T292792: 2021-10-07 network provider issues causing all Wikimedia sites to be unreachable for many users.
Thu, Oct 7, 9:01 PM · Wikimedia-Incident, SRE
Wed, Oct 6
CDanis awarded T219279: Some pages will become completely unreachable after PHP7 update due to Unicode changes a Like token.
Wed, Oct 6, 4:26 PM · MW-1.38-notes (1.38.0-wmf.2; 2021-09-28), Patch-For-Review, MW-1.35-notes (1.35.0-wmf.28; 2020-04-14), User-notice, Platform Team Workboards (Clinic Duty Team), MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), serviceops, SRE, PHP 7.2 support, MediaWiki-General
CDanis awarded T219279: Some pages will become completely unreachable after PHP7 update due to Unicode changes a Dislike token.
Wed, Oct 6, 4:26 PM · MW-1.38-notes (1.38.0-wmf.2; 2021-09-28), Patch-For-Review, MW-1.35-notes (1.35.0-wmf.28; 2020-04-14), User-notice, Platform Team Workboards (Clinic Duty Team), MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), serviceops, SRE, PHP 7.2 support, MediaWiki-General
Mon, Sep 27
CDanis edited P17330 Example SLO calculation in PromQL: hypothetical Mediawiki average latency SLO.
Mon, Sep 27, 5:02 PM
CDanis created P17330 Example SLO calculation in PromQL: hypothetical Mediawiki average latency SLO.
Mon, Sep 27, 5:01 PM
Thu, Sep 23
CDanis updated subscribers of T234450: Special:Contributions requests with a high &limit= caused excessive database load.
For posterity:
Thu, Sep 23, 8:24 PM · MW-1.31-release-notes, MW-1.33-notes, MW-1.34-notes, Platform Engineering, Security, MW-1.35-notes (1.35.0-wmf.5; 2019-11-05), User-notice, Vuln-DoS, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error
Mon, Sep 20
CDanis awarded T286639: Move Varnish test infrastructure from Vagrant to Docker a Love token.
Mon, Sep 20, 2:57 PM · good first task, SRE, Traffic
Sep 14 2021
CDanis added a comment to T290425: statograph_post service fail on alert hosts.
Because the only data available in the API is the timestamp of the most recent successfully-processed datapoint, this unfortunately left a gap in the data for two metrics:
Sep 14 2021, 7:59 PM · User-jbond, SRE-OnFire, observability, SRE
Aug 21 2021
Aklapper awarded T257066: Extension:Score / Lilypond is disabled on all wikis a Love token.
Aug 21 2021, 8:35 PM · Patch-For-Review, MW-1.36-notes (1.36.0-wmf.26; 2021-01-12), User-notice, Security-Team, Security, WMF-General-or-Unknown, MediaWiki-extensions-Score, SRE
Aug 19 2021
Ruthven awarded T257066: Extension:Score / Lilypond is disabled on all wikis a Cookie token.
Aug 19 2021, 7:56 PM · Patch-For-Review, MW-1.36-notes (1.36.0-wmf.26; 2021-01-12), User-notice, Security-Team, Security, WMF-General-or-Unknown, MediaWiki-extensions-Score, SRE
Gwendal awarded T257066: Extension:Score / Lilypond is disabled on all wikis a Party Time token.
Aug 19 2021, 6:30 PM · Patch-For-Review, MW-1.36-notes (1.36.0-wmf.26; 2021-01-12), User-notice, Security-Team, Security, WMF-General-or-Unknown, MediaWiki-extensions-Score, SRE
Bodhisattwa awarded T257066: Extension:Score / Lilypond is disabled on all wikis a Love token.
Aug 19 2021, 6:17 PM · Patch-For-Review, MW-1.36-notes (1.36.0-wmf.26; 2021-01-12), User-notice, Security-Team, Security, WMF-General-or-Unknown, MediaWiki-extensions-Score, SRE
valerio.bozzolan awarded T257066: Extension:Score / Lilypond is disabled on all wikis a Mountain of Wealth token.
Aug 19 2021, 5:07 PM · Patch-For-Review, MW-1.36-notes (1.36.0-wmf.26; 2021-01-12), User-notice, Security-Team, Security, WMF-General-or-Unknown, MediaWiki-extensions-Score, SRE
Jun 29 2021
CDanis added a comment to T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
To aid in the Puppetization and deployment: P16741 is a private-to-SRE paste that contains a configuration file suitable for use in production.
Jun 29 2021, 5:16 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis added a comment to T202061: Implement an accurate and easy to understand status page for all wikis.
In T202061#7176114, @CDanis wrote:
In T202061#7175886, @Legoktm wrote:
https://wikimedia.statuspage.io/ is live and will continue to see more development, usage and integration. Further efforts to be tracked in the Observability Workboard.
Is there documentation about this somewhere? E.g. what does "Editing" mean, which API is "API" about, what developer tools are being watched, etc.?
The section headers are not finalized -- honestly I'm not happy with them, despite having chose them. These were just the ideas that we hated the least. Documentation to be written, likely after some more discussion and some experience.
Jun 29 2021, 5:13 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
CDanis updated the task description for T285769: Ensure SRE team has a good understanding of how & when to declare an outage on the status page; & it is easy to do so.
Jun 29 2021, 5:05 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
CDanis added a comment to T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
As a fun curiosity, here's today's datacenter switchover as shown by Statuspage:
Jun 29 2021, 4:57 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis added a comment to T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
As a fun curiosity, here's today's datacenter switchover as shown by Statuspage:
Jun 29 2021, 4:57 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis created T285769: Ensure SRE team has a good understanding of how & when to declare an outage on the status page; & it is easy to do so.
Jun 29 2021, 2:55 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
Jun 28 2021
CDanis updated the task description for T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Jun 28 2021, 8:28 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
Jun 27 2021
CDanis triaged T285634: June 2021: appservers accumulating active php-fpm workers, requiring rolling restarts to avoid user-visible latency impact as High priority.
Jun 27 2021, 4:30 PM · MW-1.37-notes (1.37.0-wmf.11; 2021-06-21), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)​, wdwb-tech, Wikidata, Train Deployments, Release, Wikimedia-Incident, SRE, serviceops
CDanis created T285634: June 2021: appservers accumulating active php-fpm workers, requiring rolling restarts to avoid user-visible latency impact.
Jun 27 2021, 4:30 PM · MW-1.37-notes (1.37.0-wmf.11; 2021-06-21), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)​, wdwb-tech, Wikidata, Train Deployments, Release, Wikimedia-Incident, SRE, serviceops
Jun 25 2021
CDanis updated the task description for T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Jun 25 2021, 6:02 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis added a project to T202061: Implement an accurate and easy to understand status page for all wikis: SRE-OnFire.
Jun 25 2021, 5:39 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
CDanis added a project to T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display: SRE-OnFire.
Jun 25 2021, 5:39 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis updated the task description for T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Jun 25 2021, 5:37 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
CDanis created T285569: Automated uploads of minimal & comprehensible timeseries metrics for statuspage display.
Jun 25 2021, 5:33 PM · User-jbond, SRE-OnFire, Patch-For-Review, observability, SRE
Jun 24 2021
CDanis added a comment to T202061: Implement an accurate and easy to understand status page for all wikis.
In T202061#7175886, @Legoktm wrote:
In T202061#7175872, @lmata wrote:
https://wikimedia.statuspage.io/ is live and will continue to see more development, usage and integration. Further efforts to be tracked in the Observability Workboard.
Is there documentation about this somewhere? E.g. what does "Editing" mean, which API is "API" about, what developer tools are being watched, etc.?
Jun 24 2021, 9:10 PM · SRE Observability (FY2021/2022-Q2), SRE-OnFire, SRE
CDanis placed T267714: ripe-atlas-codfw is down up for grabs.
Unfortunately I won't have time to work on this before going on leave, but it seems like it might not be a bad task for @cmooney to learn some about RIPE Atlas. Or failing that perhaps @ayounsi will have some time for it.
Jun 24 2021, 6:19 PM · Infrastructure-Foundations, netops, SRE
May 27 2021
CDanis created P16232 Command-Line Input.
May 27 2021, 3:00 PM
May 26 2021
CDanis added a comment to T283614: RIPE Atlas monitoring of reachability & latency towards anycasted Wikidough IP.
RIPE Atlas probes support sending DoT queries; however, the option is not exposed anywhere in the measurement creation web UI, nor in the official ripe-atlas-tools distribution. The unofficial blaeu tools for creating one-off measurements support sending the option, and there is also a pull request against ripe-atlas-tools (lingering almost two years now).
May 26 2021, 6:28 PM · Traffic
CDanis created T283744: codfw appserver latency alerts flapping.
May 26 2021, 4:55 PM · serviceops
May 25 2021
CDanis added a comment to T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them.
Totally agree with all of the above! Filed a placeholder task: T283614: RIPE Atlas monitoring of reachability & latency towards anycasted Wikidough IP
May 25 2021, 4:47 PM · Infrastructure-Foundations, netops, observability, SRE
CDanis created T283614: RIPE Atlas monitoring of reachability & latency towards anycasted Wikidough IP.
May 25 2021, 4:45 PM · Traffic
May 24 2021
CDanis added a comment to T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them.
Some very initial results running against the anycast NS at https://atlas.ripe.net/frames/measurements/30366093#!probes
May 24 2021, 8:59 PM · Infrastructure-Foundations, netops, observability, SRE
CDanis added a comment to T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them.
Thanks very much for all the helpful comments!
May 24 2021, 6:09 PM · Infrastructure-Foundations, netops, observability, SRE
May 21 2021
CDanis renamed T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them from Create RIPE Atlas measurements against our public DNS servers; alert on them to Create RIPE Atlas measurements against our authoritative DNS servers; alert on them.
May 21 2021, 2:34 PM · Infrastructure-Foundations, netops, observability, SRE
CDanis renamed T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them from Create RIPE Atlas probes against our public DNS servers; alert on them to Create RIPE Atlas measurements against our public DNS servers; alert on them.
May 21 2021, 2:34 PM · Infrastructure-Foundations, netops, observability, SRE
CDanis created T283359: Create RIPE Atlas measurements against our authoritative DNS servers; alert on them.
May 21 2021, 2:33 PM · Infrastructure-Foundations, netops, observability, SRE
CDanis added a comment to T283175: Should WMCS be getting CF protection?.
Right now the WMCS NSes ({ns0,ns1}.openstack.eqiad1.wikimediacloud.org) are behind CF, but nothing else from WMCS is.
May 21 2021, 2:03 PM · SRE, SecTeam-Processed, Traffic, Cloud-Services, cloud-services-team (Kanban), Security, Security-Team
May 19 2021
CDanis triaged T283163: BGP Policy on aggregate routes prevents them being created in some circumstances. as Medium priority.
May 19 2021, 3:09 PM · Infrastructure-Foundations, SRE, netops, Traffic
May 14 2021
CDanis updated the title for P15967 ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕛☕ curl -s -v --proxy http://webproxy.eqiad.wmnet:8080 https://api.cloudflare.com >/dev/null from Masterwork From Distant Lands to ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕛☕ curl -s -v --proxy http://webproxy.eqiad.wmnet:8080 https://api.cloudflare.com >/dev/null.
May 14 2021, 4:07 PM
May 10 2021
CDanis added a comment to T275234: TATA SKY Broadband (AS134674) issues with connecting to upload.wikimedia.org.
There is evidence in our NEL data that suggests this problem still exists.
May 10 2021, 8:08 PM · Infrastructure-Foundations, SRE, netops
May 5 2021
CDanis updated the task description for T257527: automatically collect network error reports from users' browsers (Network Error Logging API).
May 5 2021, 6:08 PM · Patch-For-Review, Product-Data-Infrastructure, SRE, Goal, Epic
CDanis updated subscribers of T279380: Add Traffic's notion of "from public cloud" to Analytics webrequest data.
@fdans @JAllemandou New map entry should be ready for Analytics to set up in Turnilo :)
May 5 2021, 6:06 PM · Analytics-Kanban, SRE, Analytics, Traffic
CDanis updated the task description for T279380: Add Traffic's notion of "from public cloud" to Analytics webrequest data.
May 5 2021, 6:05 PM · Analytics-Kanban, SRE, Analytics, Traffic
Apr 27 2021
Ladsgroup awarded T250205: Reduce rate of purges emitted by MediaWiki a Love token.
Apr 27 2021, 9:49 AM · Platform Engineering Roadmap Decision Making, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Sustainability (Incident Followup), Performance-Team (Radar), serviceops, Traffic, SRE
Apr 26 2021
CDanis edited P13393 eet.py.
Apr 26 2021, 7:27 PM
CDanis added a comment to T281055: mr1 port utilization alerts shouldn't mention hash page in their IRC logs.
Moving to AM sounds good to me. But if needed, in the interim we could change the magic string we use in check_librenms to something else instead of hash page, which I chose for simplicity but has maybe just caused more confusion.
Apr 26 2021, 4:23 PM · Infrastructure-Foundations, SRE, netops
Apr 15 2021
CDanis committed rOHPUd9843950d162: Revert "prepend esams/knams" (authored by CDanis).
Revert "prepend esams/knams"
Apr 15 2021, 3:40 PM
CDanis added a reverting change for rOHPU43f3d38d6d2f: prepend esams/knams: rOHPUd9843950d162: Revert "prepend esams/knams".
Apr 15 2021, 3:40 PM
Apr 14 2021
CDanis committed rOHPU43f3d38d6d2f: prepend esams/knams (authored by CDanis).
prepend esams/knams
Apr 14 2021, 9:51 PM
CDanis updated the task description for T257527: automatically collect network error reports from users' browsers (Network Error Logging API).
Apr 14 2021, 9:42 PM · Patch-For-Review, Product-Data-Infrastructure, SRE, Goal, Epic
Apr 7 2021
JAllemandou awarded T279380: Add Traffic's notion of "from public cloud" to Analytics webrequest data a Baby Tequila token.
Apr 7 2021, 7:58 AM · Analytics-Kanban, SRE, Analytics, Traffic
Apr 5 2021
CDanis created T279380: Add Traffic's notion of "from public cloud" to Analytics webrequest data.
Apr 5 2021, 8:02 PM · Analytics-Kanban, SRE, Analytics, Traffic
Mar 31 2021
CDanis added a comment to T279013: Phabricator intermittently slow; db connection failures to m3-master.eqiad.wmnet with "Temporary failure in name resolution".
I'm not sure if we tend to use IP addresses directly in Mediawiki out of latency concerns, reliability concerns, PHP/other client bugginess concerns, or DNS recursor capacity concerns, or some mix of all of the above. (I've seen all of the above occur at different times.)
Mar 31 2021, 9:25 PM · serviceops, User-brennen, Phabricator
Mar 15 2021
CDanis updated the task description for T277485: Include request_id in the comments of database queries.
Mar 15 2021, 5:33 PM · Sustainability (Incident Followup), Performance-Team, Platform Team Workboards (Clinic Duty Team), Wikimedia-Rdbms, serviceops-radar
CDanis added a subtask for T277417: 14 March 2021 Wikimedia API Outage: T277485: Include request_id in the comments of database queries.
Mar 15 2021, 4:53 PM · Wikimedia-Incident, SRE
CDanis added a parent task for T277485: Include request_id in the comments of database queries: T277417: 14 March 2021 Wikimedia API Outage.
Mar 15 2021, 4:53 PM · Sustainability (Incident Followup), Performance-Team, Platform Team Workboards (Clinic Duty Team), Wikimedia-Rdbms, serviceops-radar
CDanis created T277485: Include request_id in the comments of database queries.
Mar 15 2021, 4:53 PM · Sustainability (Incident Followup), Performance-Team, Platform Team Workboards (Clinic Duty Team), Wikimedia-Rdbms, serviceops-radar
Mar 10 2021
CDanis closed T263496: Augment NEL reports with GeoIP country code and network AS number as Resolved.
ASN, ISP/organization, country, & subdivision are now visible in Logstash!
Mar 10 2021, 9:36 PM · Patch-For-Review, Analytics, SRE
CDanis closed T263496: Augment NEL reports with GeoIP country code and network AS number, a subtask of T257527: automatically collect network error reports from users' browsers (Network Error Logging API), as Resolved.
Mar 10 2021, 9:35 PM · Patch-For-Review, Product-Data-Infrastructure, SRE, Goal, Epic
CDanis created T277096: WMF helmfile installation does not work for ZSH users.
Mar 10 2021, 8:42 PM · Kubernetes, serviceops
CDanis updated the task description for T276486: gerrit's sshd is incompatible with RSA pubkeys + Fedora 33 clients (and future versions of OpenSSH proper).
Mar 10 2021, 3:12 PM · Gerrit
CDanis updated the task description for T276486: gerrit's sshd is incompatible with RSA pubkeys + Fedora 33 clients (and future versions of OpenSSH proper).
Mar 10 2021, 3:10 PM · Gerrit
Mar 8 2021
CDanis added a comment to T276486: gerrit's sshd is incompatible with RSA pubkeys + Fedora 33 clients (and future versions of OpenSSH proper).
In T276486#6891521, @hashar wrote:
What I understand is that the Mina SSHD version embedded in our Gerrit does not have support for the rsa-sha-xxx and would eventually need the version it ships would need to be upgraded.
Mar 8 2021, 2:21 PM · Gerrit
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL