Log In
T147199
Removing support for DES-CBC3-SHA TLS cipher (drops IE8-on-XP support)
Closed, ResolvedPublic
Assigned To
BBlack
Authored By
BBlack
Oct 3 2016, 2:52 PM
Tags
Traffic (TLS)
SRE (Backlog)
Patch-For-Review
User-notice (Already announced/Archive)
Browser-Support-Internet-Explorer (IE8)
Subscribers
Aklapper
Bawolff
BBlack
Chmarkine
csteipp
Dzahn
Elitre
View All 27 Subscribers
Tokens
Description
This is one of our two last remaining non-forward-secret ciphers, which we'd like to eliminate as soon as reasonably possible. It's also the subject of the SWEET32 birthday attack due to being a 64-bit (block size) cipher, which we've mostly-mitigated in other ways for now (shortened session key lifetimes).
The only statistically-significant browser which relies on this cipher to communicate with us is IE8-on-XP, which is long-unsupported (over 2 years now) and horribly insecure, so any motivation we can give users to get off of this browser is a net win for everyone involved.
Currently this cipher amounts to ~0.17% of all requests to our sites (and has been slowly declining for some time), and we've been running a very limited campaign for a while now which redirects a very small percentage of those requests (filtered down to just /wiki/ pageviews on the desktop sites, and only 1% odds) to the information at https://wikitech.wikimedia.org/wiki/HTTPS:_Browser_Recommendations . Because of technical limitations, we can't scale up that redirect much without having a better mechanism in place. IE8-on-XP is too old for CentralNotice JS to function correctly, so CN isn't really an option for campaigning here.
Users which cannot move off of the underlying Windows XP operating system can install the latest Firefox easily and use that to connect to us with much more secure cipher choices, so there is a fairly painless path forward for these users.
The plan of action here is this:
- Coordinate with the Community team to ensure they're aware of everything here ahead of any user complaints. This probably isn't the kind of situation where pre-announcements on community talk pages and/or mailing lists help much, as the target users aren't likely to be readers there, but it's still better to be prepared with answers.
- Prepare a Varnish synthetic page output based on our standard error page templates, which gives a very quick note about connection security and offers a link to the explanatory https://wikitech.wikimedia.org/wiki/HTTPS:_Browser_Recommendations .
- Code to show this to a random X% of pageviews from affected clients (already implemented for redirect, needs switch to synthetic output above).
- Once the above is ready, we'll set the final timeline in place: a 2 month period over which we'll ramp the percentage up from a small value (say, 5% of affected pageviews) to 100% of affected pageviews, and a further month during which we'll still allow connections from affected clients, but 100% of their pageviews will go to the information page.
- After the 3 month window is complete, remove support for this cipher entirely.
Details
ProjectSubject
operations/puppetvcl: remove 3DES deprecation warning
operations/puppetssl_ciphersuite: dump 3DES on 2017-11-17
operations/puppetuse synthetic warning for 1% of 3DES pageviews
Customize query in gerrit
Related Objects
Task Graph
Mentions
StatusAssignedTask
ResolvedBBlackT118181 Planning for phasing out non-Forward-Secret TLS ciphers
OpenNoneT147967 The WMF-Last-Access Set-Cookie header should follow RFC 2965 syntax rather than the pre-RFC Netscape format
ResolvedBBlackT147199 Removing support for DES-CBC3-SHA TLS cipher (drops IE8-on-XP support)
ResolvedJohanT163251 Communicate dropping IE8-on-XP support (a security change) to affected editors and other community members
ResolvedJohanT172418 Get translations for "IE8 on XP won't work"
There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application removed a project: Patch-For-Review. · View Herald Transcript
Oct 3 2016, 2:52 PM
BBlack moved this task from Triage to TLS on the Traffic board.
Oct 3 2016, 2:55 PM
BBlack updated the task description. (Show Details)
Oct 3 2016, 3:01 PM
BBlack mentioned this in T147202: Removing support for AES128-SHA TLS cipher.
Oct 3 2016, 3:05 PM
BBlack updated the task description. (Show Details)
Oct 3 2016, 3:46 PM
ema added a subscriber: ema.
Oct 5 2016, 3:43 PM
BBlack mentioned this in T147967: The WMF-Last-Access Set-Cookie header should follow RFC 2965 syntax rather than the pre-RFC Netscape format.
Oct 12 2016, 8:47 PM
BBlack added a subtask: T147967: The WMF-Last-Access Set-Cookie header should follow RFC 2965 syntax rather than the pre-RFC Netscape format.
Legoktm renamed this task from Removing support for DES-CBC3-SHA TLS cipher to Removing support for DES-CBC3-SHA TLS cipher (drops IE8-on-XP support).
Oct 12 2016, 9:23 PM
BBlack mentioned this in T144194: Varnish-triggered CN campaign about browser security.
Oct 13 2016, 2:51 AM
JW added a subscriber: JW.
Oct 27 2016, 4:42 AM
BBlack updated the task description. (Show Details)
Oct 31 2016, 11:12 PM
After some IRC discussion, it seemed better to host the content pages on metawiki. It will look more-official, and it will also be easier to develop and maintain the content. Description updated to reflect that.
BBlack added a comment.
Nov 15 2016, 3:53 PM
We've been stalled out on this because ETOOBUSY, which continues to push off the start of the timeline as well. At best we're looking at the 3-month window starting ~Dec1 at this point, and that's if we put some priority back on this.
BBlack added a comment.
Mar 31 2017, 4:04 PM
We've been stalling on this a bit too long now. I'd like to start kicking off this process and getting in touch with Community as well. I've kinda backtracked on the idea of a redirect to meta-wiki for the initial notice to the user. I think for the initial page replacement, we should stick with a synthetic output directly from Varnish itself. That output can in turn contain a link to our existing wikitech page, or a similar page to that one on meta-wiki that's more-detailed. I've stolen from our existing Varnish errorpage.html and proposed some HTML for this here: P5175 (I wish pastes could be viewed raw with content-type!). Obviously wording and layout can be worked on a bit (and actual dates inserted for the real thing), but I like it having our standardized error theme and logo.
MoritzMuehlenhoff added a subscriber: MoritzMuehlenhoff.
Apr 3 2017, 12:24 PM
MoritzMuehlenhoff added a comment.
Apr 3 2017, 12:37 PM
Text looks good to me, but two points:
BBlack added a comment.
Apr 3 2017, 2:26 PM
Depending on the context I've been flipping between whether we're talking about just 3DES or both of the non-FS ciphers, sorry. In current weekly stats, 3DES is around 0.125% and AES128-SHA is around 0.225% for total of ~0.35% non-FS ( https://grafana.wikimedia.org/dashboard/db/tls-ciphers ). Probably the bolded redirect notice with the 0.2% number should be removed from the wikitech page, and the synthetic varnish error shown here should use "less than 0.2%" during the 3DES campaign. Once 3DES is disabled we can re-assess how we approach AES128-SHA and fix up various things appropriately.
BBlack updated the task description. (Show Details)
Apr 17 2017, 2:55 PM
gerritbot added a comment.
Apr 17 2017, 3:50 PM
Change 348477 had a related patch set uploaded (by BBlack):
[operations/puppet@production] use synthetic warning for 1% of 3DES pageviews
https://gerrit.wikimedia.org/r/348477
gerritbot added a project: Patch-For-Review.
Apr 17 2017, 3:50 PM
BBlack updated the task description. (Show Details)
Apr 17 2017, 3:58 PM
gerritbot added a comment.
Apr 17 2017, 6:19 PM
Change 348477 merged by BBlack:
[operations/puppet@production] use synthetic warning for 1% of 3DES pageviews
https://gerrit.wikimedia.org/r/348477
BBlack claimed this task.
Apr 17 2017, 6:47 PM
I've deployed the change above, which gets all of the basics on track for how we want to operate the real campaign. We'll obviously make appropriate minor wording changes once we have dates set and as percentages increase.
Next step is I need to get in touch with the Community team about all of this before we move any further.
The synthetic warning output can be manually viewed at /test-sec-warning on any domain that maps to the text cluster, e.g.: https://en.wikipedia.org/test-sec-warning
Whatamidoing-WMF mentioned this in T163251: Communicate dropping IE8-on-XP support (a security change) to affected editors and other community members.
Apr 18 2017, 7:52 PM
Whatamidoing-WMF created subtask T163251: Communicate dropping IE8-on-XP support (a security change) to affected editors and other community members.
Elitre added a subscriber: Elitre.
Apr 21 2017, 10:35 AM
Jdforrester-WMF added a subscriber: Jdforrester-WMF.
Apr 27 2017, 10:40 PM
Johan added a project: User-notice.
Aug 3 2017, 2:44 AM
Johan moved this task from To Triage to In current Tech/News draft on the User-notice board.
Johan moved this task from In current Tech/News draft to Announce in next Tech/News on the User-notice board.
Johan added a subscriber: Johan.
Aug 3 2017, 4:41 AM
Since this task was created in 2015, Firefox has stopped supporting Windows XP. They might need to install Firefox 52 (as opposed to current 54). Most folks who are still using Internet Explorer 8 on XP will probably find it fairly difficult to find and install an old version of a browser.
Johan moved this task from Announce in next Tech/News to In current Tech/News draft on the User-notice board.
Aug 3 2017, 5:02 AM
Liuxinyu970226 added a subscriber: Liuxinyu970226.
Aug 3 2017, 7:48 AM
BBlack added a comment.
Aug 3 2017, 2:07 PM
Cross-ticket updates: There's a separate sub-ticket for the Communications side of this change at T163251, and a timeline has been laid out there in T163251#3478043 . The TL;DR of the timeline is we'll ramp from 5% to ~29% blocked over the period of Aug 17 -> Oct 12, then 100% blocked on Oct 17, then protocol-disabled on Nov 17.
Other relevant notes:
Bawolff added a subscriber: Bawolff.
Aug 3 2017, 5:59 PM
perhaps in the error page, the "use Firefox!" should be directly linked to the firefox 52 esr download page. The easier for users to find the link, and the less clicks the user has to go through, the more likely they will actually do it.
Quiddity added a subscriber: Quiddity.
Aug 3 2017, 8:23 PM
In T147199#3498052, @Bawolff wrote:
perhaps in the error page, the "use Firefox!" should be directly linked to the firefox 52 esr download page. The easier for users to find the link, and the less clicks the user has to go through, the more likely they will actually do it.
+1 to that. The next issue of Tech/News (draft, but will be frozen for translators in ~20 hours) points directly to the ESR page.
However, note that Firefox ESR 52 is ending support in Q2 of 2018, IIUC from the FAQ
But there are no decent alternatives for WinXP, at all.
Bawolff added a comment.
Aug 3 2017, 11:01 PM
In T147199#3498842, @Quiddity wrote:
In T147199#3498052, @Bawolff wrote:
perhaps in the error page, the "use Firefox!" should be directly linked to the firefox 52 esr download page. The easier for users to find the link, and the less clicks the user has to go through, the more likely they will actually do it.
+1 to that. The next issue of Tech/News (draft, but will be frozen for translators in ~20 hours) points directly to the ESR page.
However, note that Firefox ESR 52 is ending support in Q2 of 2018, IIUC from the FAQ
But there are no decent alternatives for WinXP, at all.
Midori maybe? Thats starting to get rather obscure though. There is a list at http://www.msfn.org/board/topic/176386-list-of-web-browsers-working-with-xp-2017/ but they mostly sound sketch.
BBlack added a comment.
Aug 3 2017, 11:20 PM
Even while FF 52 is still supported by Mozilla, it's unlikely that Mozilla's security efforts can actually prevent all the possible exploits that breach the underlying WinXP.
From the perspective of the users' own local security concerns: the "end of support" for FF52 just means updates stop coming from Mozilla and it gets harder to initially install (have to go dig for backdated release links), but doesn't necessarily denote any big downgrade in the effective security of the user (which is basically zero, so long as they stick to XP).
From our perspective there's a couple things to note about the FF52 end date:
  1. The eventual end-date for FF52 is and has been a factor on our end: it's important we make our transition beforehand, or it may get even harder to offer any legitimate-looking, immediate, and easy "install Firefox" link.
  2. FF52 as it exists today already supports the strongest standardized TLSv1.2 cipher available today (ECDHE-ECDSA-CHACHA20-POLY1305 + X25519). So even if they get no further sec updates from Mozilla, we're unlikely to have a reason to disable them for "bad crypto" reasons anytime in the foreseeable future. Apple's latest (Safari 10) is further behind the curve on multiple fronts than FF52-for-XP, and way more popular.
MaxSem added a subscriber: MaxSem.
Aug 4 2017, 6:30 PM
Pigsonthewing added a subscriber: Pigsonthewing.
Aug 7 2017, 9:59 PM
Users which cannot move off of the underlying Windows XP operating system can install the latest Firefox easily
For some value of "easily" which disregards corporate IT policies :-(
MaxSem added a comment.
Aug 7 2017, 10:34 PM
If a corporation is insane enough to still run XP and force their users to run IE, we can only hope that yet another site they can't use will be the final straw forcing them to do something.
TheDJ added a subscriber: TheDJ.
Aug 8 2017, 12:25 AM
@Pigsonthewing if such companies are out there, then us cutting them off might finally be an indication to them how incredibly outdated they are and that it is time to get with the program... I personally count on some library computers without access to Wikipedia pretty soon and that's a good thing.
RandomDSdevel awarded a token.
Aug 8 2017, 12:46 AM
Nirmos mentioned this in T136203: Drop Grade C support for IE8.
Aug 8 2017, 11:35 AM
Nirmos added a subscriber: Nirmos.
Aug 8 2017, 11:44 AM
Johan moved this task from In current Tech/News draft to Already announced/Archive on the User-notice board.
Aug 9 2017, 2:45 PM
GeoffreyT2000 added a subscriber: GeoffreyT2000.
Aug 19 2017, 2:50 PM
What about IE7 on Windows XP and Windows Vista? If this also no longer has access to Wikipedia, then we should next do IE8 and IE9 on Windows Vista and Windows 7, followed by IE10 on Windows 7 and Windows 8 (not 8.1).
Pigsonthewing added a comment.
Aug 19 2017, 4:54 PM
In T147199#3508328, @MaxSem wrote:
If a corporation is insane enough to still run XP and force their users to run IE, we can only hope that yet another site they can't use will be the final straw forcing them to do something.
Yes, Our mission is indeed to try to force corporate IT upgrades, and not to make knowledge freely available.
Nevertheless, my point stands. The claim I quoted is clearly false.
BBlack added a comment.
Aug 19 2017, 4:56 PM
Both IE7 and IE8 for XP are what's being cut off in this transition, with IE8 being the newest IE that's even available for XP, AFAIK. However, we're not doing this with the express intent of deprecating older browser tech; it's just the natural fallout of raising our minimum security level for network connections. There's no further firm plans yet on Operations' end of things regarding deprecating specific browser versions, but we will most likely run through similar cipher/protocol deprecations in the future which may take out older browsers along the way like this one did.
I can do a sort of informal brain dump of what little we know about future IE+Windows problems:
Vista: supports, AFAIK, IE7 - IE9. Vista as well as all IE versions that run on it are out-of-support from Microsoft, so we'd certainly *like* people to abandon these ASAP, but there's no driving need yet, and I haven't dug into whether stats support this being an easy transition yet. IE7/Vista will continue to work through the current transition, as it supports much better ciphers than IE7 or 8 on XP did. However, IE7/Vista only supports TLS protocol version 1.0 (no 1.1 or 1.2 support). Therefore, support for IE7/Vista will likely fall apart everywhere on the Internet by about mid-2018 as TLSv1.0 gets dropped everywhere (see more on that below). I believe even IE9/Vista will fall to the same TLSv1.0 transition period, as I don't think any version of IE before IE11 supported TLSv1.1+, regardless of OS version. There may or may not be options like FF or Chrome for Vista users at that time, I haven't looked into that part yet.
Win7-10: are all still supported by MS at the OS level, and fare much better on these fronts than XP and Vista did. I believe Win7 security patches stop in early 2020, so there's still lots of time for it to fade out a little more gracefully, too. However, there are still lots of users (esp Win7/8 users) using IE versions 9-10 on these platforms, and they'll all need to upgrade to IE11 (or Edge, FF, Chrome, etc) by about the same mid-2018 timeframe as above, for the same basic reasons (to gain TLSv1.1+ support). IE11 on all of Win7-10 supports fairly modern crypto (TLSv1.2, ECDHE w/ P-256, AES-GCM for an AEAD cipher, etc), so there probably won't be any pressing need to drop support for them for a very long time, barring fresh new attacks on what is today the second-best standardized crypto suite available to anyone (the best being ChaPoly+X25519 that modern FF and Chrome support. Both that and what IE11/Win7 supports are strong enough that they continue to be used as primitives in the upcoming TLSv1.3 protocol as well).
In general, the mid-2018 transition where the Internet kills TLSv1.0 is going to be a little rough on older UAs. Aside from everything mentioned above in the Microsoft realm (all IE<11, and implicitly all Vista), we'll also be losing all Android versions prior to v4.4 (some even higher than that, with some vendors' customized builds that lagged behind on security...) and all Safari/iOS versions prior to v7. Unlike some of the other recent transitions, we'll probably follow the trend of the rest of the Internet on this one instead of trying to help drive the effort ourselves. It will be driven by commercial sites processing payment transactions that must drop TLSv1.0 by mid-2018 to keep their PCI compliance (and thus our Fundraising sites as well, but not necessarily the wikis).
Krenair added a subscriber: Krenair.
Aug 19 2017, 5:02 PM
BBlack added a comment.
Aug 19 2017, 5:08 PM
In T147199#3535264, @Pigsonthewing wrote:
In T147199#3508328, @MaxSem wrote:
If a corporation is insane enough to still run XP and force their users to run IE, we can only hope that yet another site they can't use will be the final straw forcing them to do something.
Yes, Our mission is indeed to try to force corporate IT upgrades, and not to make knowledge freely available.
Nevertheless, my point stands. The claim I quoted is clearly false.
It's not our mission to force corporate IT upgrades, but it is clearly on them rather than us to keep up with the most basic of upgrades. An IT department is supposedly staffed by professionals who have taken on the duty of maintaining systems for end-users. If anything, they should be held to higher standards than individual end-users at home. If the end user at home can install Firefox, so can an IT department. If the IT department in question has completely abandoned its users, that's an entirely separate problem that organization must face for a variety of reasons unrelated to this, and it's not our responsibility to take into account any other organizations' internal failures when considering our policies and actions.
Our mission is to make knowledge freely available to the world. It is also implicitly our mission to do so without irresponsibly subjecting our users to surveillance and/or censorship of their reading habits by state and/or commercial actors. It is on these principles that we advance the minimum security requirements we enforce. We're not asking anyone to be bleeding edge here. We're asking them to stop using platforms that are horribly insecure and outdated. In-between what we're dropping support for today and the bleeding edge of the most-current software, there exists several generations of slightly-less-horribly-insecure software which we continue to support which they can choose to upgrade to (or could have years ago).
MichaelSchoenitzer awarded a token.
Aug 31 2017, 8:45 PM
Framawiki added a subscriber: Framawiki.
Aug 31 2017, 10:00 PM
Nirmos mentioned this in T25932: Allow use of semantic HTML5 elements in wikitext.
Sep 14 2017, 4:40 PM
Liuxinyu970226 awarded a token.
Oct 9 2017, 2:32 PM
gerritbot added a comment.
Oct 16 2017, 7:00 PM
Change 384578 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] ssl_ciphersuite: dump 3DES on 2017-11-17
https://gerrit.wikimedia.org/r/384578
Pigsonthewing added a comment.
Oct 16 2017, 7:07 PM
Sad to see this claim made in 'Tech News' today:
If you use Internet Explorer 8 on Windows XP you can install Firefox 52 ESR instead
.
Johan added a comment.
Oct 16 2017, 7:15 PM
@Pigsonthewing Yes, Tech News deals in simplification, for a number of reasons (non-native speakers without access to a translation to their language, non-technical users, ease of translation etc etc). This comes at the cost of precision, in this case "you" instead of "you or someone with administrator access". The newsletter makes similar simplified claims most weeks, I'd say.
Pigsonthewing added a comment.
Edited · Oct 16 2017, 9:02 PM
simplification... at the cost of precision
"you" instead of "you or someone with administrator access".
That's an unnecessarily over-verbose example (and still false in some cases); all that is needed is, say, "you may be able to install" instead of "you can install". Alternatively, "try installing", which is one character shorter.
Johan added a comment.
Oct 19 2017, 4:22 PM
Very well. "You may be able to install" is actually fairly complicated to parse (three verbs, not obvious to non-fluent speakers how they work together), but it might have been phrased better. This, however, is not on anyone you've discussed this with earlier in the thread, as Tech News items are typically phrased by me and not the developers. I suggest further discussion about how items are phrased in Tech News take place either on m:Talk:Tech/News/2017/42 or on m:User talk:Johan (WMF).
Whatamidoing-WMF mentioned this in T97701: Can't visit login page or https wiki pages in IE7/8 on Windows XP (on SauceLabs).
Oct 27 2017, 5:44 PM
MattFitzpatrick added a project: Browser-Support-Internet-Explorer​.
Nov 3 2017, 3:28 PM
MattFitzpatrick moved this task from Unsorted to IE8 on the Browser-Support-Internet-Explorer board.
gerritbot added a comment.
Nov 17 2017, 1:06 PM
Change 384578 merged by BBlack:
[operations/puppet@production] ssl_ciphersuite: dump 3DES on 2017-11-17
https://gerrit.wikimedia.org/r/384578
BBlack closed this task as Resolved.
Nov 17 2017, 3:05 PM
No 3DES connections or saved sessions left on the public cache endpoints \o/
BBlack mentioned this in T180792: Remove 3DES patch from OpenSSL builds.
Nov 17 2017, 3:06 PM
Dzahn awarded a token.
Nov 17 2017, 3:07 PM
BBlack closed subtask T163251: Communicate dropping IE8-on-XP support (a security change) to affected editors and other community members as Resolved.
Nov 17 2017, 3:48 PM
Liuxinyu970226 removed a subscriber: Liuxinyu970226.
Nov 24 2017, 11:54 AM
Izno added a subscriber: Izno.
Edited · Dec 28 2017, 1:22 PM
Just for documentation's sake, this ended AutoWikiBrowser on XP (ref request for help at en.wp). Users will either need to upgrade their Windows operating system or attempt to run it on Linux or Mac to use AWB on Wikimedia wikis.
BBlack mentioned this in T185582: Wikipedia no longer accessible to those using some braille devices.
Jan 26 2018, 6:44 PM
BBlack updated the task description. (Show Details)
May 4 2018, 6:05 PM
Vgutierrez mentioned this in T196371: Provide a multi-language user-faced warning regarding AES128-SHA deprecation.
Jun 4 2018, 2:09 PM
gerritbot added a comment.
Jun 13 2018, 10:01 AM
Change 440087 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] vcl: remove 3DES deprecation warning
https://gerrit.wikimedia.org/r/440087
gerritbot added a comment.
Jun 13 2018, 4:04 PM
Change 440087 abandoned by Ema:
vcl: remove 3DES deprecation warning
Reason:
There's no rush to merge this, let's wait for https://gerrit.wikimedia.org/r/#/c/operations/puppet/ /440114/ instead
https://gerrit.wikimedia.org/r/440087
Jdforrester-WMF removed a subtask: T147967: The WMF-Last-Access Set-Cookie header should follow RFC 2965 syntax rather than the pre-RFC Netscape format.
Aug 7 2018, 6:44 PM
Jdforrester-WMF added a parent task: T147967: The WMF-Last-Access Set-Cookie header should follow RFC 2965 syntax rather than the pre-RFC Netscape format.
Aklapper mentioned this in T178356: Raise Grade A JavaScript requirement from ES5 (2009) to ES6 (2015).
Aug 18 2020, 7:40 AM
Log In to Comment
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL