Introduce non-voting jobs with quibble+apache
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	awight
	Mar 4 2021, 9:44 AM

Description

Quibble defaults to the PHP built-in server, but since release 0.0.46 also includes Apache support. The quibble-apache:0.0.1 image includes the necessary packages.

In this task, we'll create a few non-voting jobs in various repos, which exercise quibble+apache. We want to determine:

How does performance compare to the regular quibble job?
Do some repos have quirks to address before we can use the apache job?

See this example job which includes the necessary glue. Hopefully this can be simplified a bit.

The major difference is that the apache job can serve requests concurrently, which will expose some issues with our code and require fixes to stabilize tests.

In the long-term, if the results are encouraging we might decide to switch all quibble jobs to use apache.

Details

Subject	Repo	Branch	Lines +/-
EntityDataRequestHandler: Remove ob_clean() call	mediawiki/extensions/Wikibase	REL1_36	+0 -2
WIP DNM: Selenium: videos for repo & bridge using wdio-video-reporter	mediawiki/extensions/Wikibase	master	+21 K -5 K
selenium: Restart php7.2-fpm after modifying LocalSettings.php	mediawiki/extensions/GrowthExperiments	master	+13 -0
EntityDataRequestHandler: Remove ob_clean() call	mediawiki/extensions/Wikibase	REL1_35	+0 -2
EntityDataRequestHandler: Remove ob_clean() call	mediawiki/extensions/Wikibase	master	+0 -2
zuul: Add Apache/Selenium job to experimental group for GrowthExperiments and VE	integration/config	master	+4 -0
zuul: Use non voting wmf-quibble-apache-selenium-php72-docker for Wikibase	integration/config	master	+2 -1
jjb: Add quibble apache job for selenium too	integration/config	master	+11 -0
jjb: wikimedia Quibble job with Apache	integration/config	master	+10 -1
Zuul: [mediawiki/extensions/Wikibase] Add non-voting quibble-apache job	integration/config	master	+8 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min
Open	None	T226869 Run browser tests in parallel
Resolved	None	T285649 Switch all Quibble Selenium and api-testing jobs to use apache
Resolved	None	T276428 Introduce non-voting jobs with quibble+apache
Resolved	awight	T225218 Consider httpd for quibble instead of php built-in server
Open	None	T199393 Reproducible deadlock in User::addToDatabase() when api.php?action=createaccount is called simultaneously by several users
Resolved	hashar	T342088 Run Quibble Mariadb server with innodb_print_all_deadlocks
Resolved	kostajh	T282154 Wikibase: Special:EntityData compression encoding issues in local dev env

Event Timeline

awight created this task.Mar 4 2021, 9:44 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 4 2021, 9:44 AM

awight added a subtask: T225218: Consider httpd for quibble instead of php built-in server.Mar 4 2021, 9:45 AM

kostajh subscribed.Mar 4 2021, 9:47 AM

hashar mentioned this in T277202: Can we make the quibble webserver used in CI log the time it has taken to execute calls to the API?.Mar 11 2021, 6:00 PM

The major difference is that the apache job can serve requests concurrently, which will expose some issues with our code and require fixes to stabilize tests.

Have we seen this kind of issue happening in practice, or is it a hunch? If a hunch, it might be worth just flipping the switch for all selenium jobs and see what happens. I struggle to imagine real examples where using Apache could cause tests to start failing, especially with local development mostly happening in VMs or containers that have always used Apache or Nginx with concurrency, and people do run them locally and see it pass.

Addshore subscribed.Mar 11 2021, 7:35 PM

In T277202#6905923, @hashar wrote:

But that opens some other issues such as race conditions between queries (and potentially locking issue with Sqlite).

Hmm, where is that perceived to be an issue?
I don't think we would run into anything like that for our tests for example.
Setup is generally done serially anyway, the current pinch point seems to be API calls that happen when the UI is actually loaded, and these seem to cause many timeouts! :(

T199393#6702427 is an example, and I am pretty sure we had issue when multiple API calls triggered a localisation cache rebuild which raced for a lock on the sqlite database (got solved by always generating the loalisation cache when installing mediawiki). There must be a bunch of other related races. But we will need to have the jobs and run them to surface them.

Off the top of my head there shouldn't be any more wide spread issues with concurrent requests.
If tests needs to do things in a certain order, they should make sure they do them in a certain order.

I volunteer Wikibase to trial some non voting jobs running this apache varient

In T276428#6906218, @Krinkle wrote:

The major difference is that the apache job can serve requests concurrently, which will expose some issues with our code and require fixes to stabilize tests.

Have we seen this kind of issue happening in practice, or is it a hunch? If a hunch, it might be worth just flipping the switch for all selenium jobs and see what happens. I struggle to imagine real examples where using Apache could cause tests to start failing, especially with local development mostly happening in VMs or containers that have always used Apache or Nginx with concurrency, and people do run them locally and see it pass.

Yeah, this was one issue (T199393) spotted during test runs (worked around with https://gerrit.wikimedia.org/r/c/mediawiki/core/+/654417) and another one was https://gerrit.wikimedia.org/r/c/654443

I am in favor of flipping the switch too, if we can have some people around to help fix tests quickly.

Change 670965 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] Add php72-apache flavour to quibble jobs

https://gerrit.wikimedia.org/r/670965

gerritbot added a project: Patch-For-Review.Mar 11 2021, 10:22 PM

Change 670967 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] non voting quibble-apache job for Wikibase

https://gerrit.wikimedia.org/r/670967

Change 670965 merged by jenkins-bot:
[integration/config@master] jjb: wikimedia Quibble job with Apache

https://gerrit.wikimedia.org/r/670965

Change 670967 merged by jenkins-bot:
[integration/config@master] Zuul: [mediawiki/extensions/Wikibase] Add non-voting quibble-apache job

https://gerrit.wikimedia.org/r/670967

Maintenance_bot removed a project: Patch-For-Review.Mar 12 2021, 2:10 PM

Mentioned in SAL (#wikimedia-releng) [2021-03-12T14:10:53Z] <addshore> reload zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/670967/ T276428

kostajh awarded a token.Mar 12 2021, 2:13 PM

Change 671167 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] jjb: Add quibble apache job for selenium too

https://gerrit.wikimedia.org/r/671167

Change 671168 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] zuul: Use non voting wmf-quibble-apache-selenium-php72-docker for Wikibase

https://gerrit.wikimedia.org/r/671168

Change 671167 merged by jenkins-bot:
[integration/config@master] jjb: Add quibble apache job for selenium too

https://gerrit.wikimedia.org/r/671167

Change 671168 merged by jenkins-bot:
[integration/config@master] zuul: Use non voting wmf-quibble-apache-selenium-php72-docker for Wikibase

https://gerrit.wikimedia.org/r/671168

Here is a lovely run with many failures! :D
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2/console

Change 671188 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] WIP DNM: data-bridge: dump browser console logs on failure

https://gerrit.wikimedia.org/r/671188

So it looks like every non voting selenium job for Wikibase using apache as the webserver has failed since being introduced.
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/buildTimeTrend

A quick scan seems to show that most of these are timeout waiting for elements to be displayed.

Looking at the most recent failure, where the current job passed https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/47/console
The first issue is:

20:00:55 [0-1] Error in "item can add a statement using the keyboard"
20:00:55 element ("div.wikibase-addtoolbar > .wikibase-toolbar-button-add > a") still not displayed after 10000ms

I played around with https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/671188 a little more testing videos and trying to log console logs to CI but generally failed and couldn;t debug the failing tests any further..

Will have to try another approach next (maybe locally)

kostajh mentioned this in T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Jun 9 2021, 7:18 AM

kostajh mentioned this in T284165: Create the first VisualEditor Selenium test.Jun 22 2021, 9:04 PM

Change 701050 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] zuul: Add quibble-apache-selenium job to experimental group for GrowthExperiments

https://gerrit.wikimedia.org/r/701050

Change 701050 merged by jenkins-bot:

[integration/config@master] zuul: Add Apache/Selenium job to experimental group for GrowthExperiments and VE

https://gerrit.wikimedia.org/r/701050

kostajh mentioned this in T285649: Switch all Quibble Selenium and api-testing jobs to use apache.Jun 28 2021, 9:02 AM

In T276428#6916188, @Addshore wrote:
So it looks like every non voting selenium job for Wikibase using apache as the webserver has failed since being introduced.
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/buildTimeTrend

A quick scan seems to show that most of these are timeout waiting for elements to be displayed.

Looking at the most recent failure, where the current job passed https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/47/console
The first issue is:
20:00:55 [0-1] Error in "item can add a statement using the keyboard"
20:00:55 element ("div.wikibase-addtoolbar > .wikibase-toolbar-button-add > a") still not displayed after 10000ms
I played around with https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/671188 a little more testing videos and trying to log console logs to CI but generally failed and couldn;t debug the failing tests any further..

Will have to try another approach next (maybe locally)

I can't reproduce the error locally.

I'd be interested to look at the VM running the tests and just see if it is even more stuff like T281122: Wikibase selenium tests timeout, seemingly due to "memory compaction" events on CI VMs and VM memory compaction etc making things freeze up

In T276428#7183437, @Addshore wrote:

I'd be interested to look at the VM running the tests and just see if it is even more stuff like T281122: Wikibase selenium tests timeout, seemingly due to "memory compaction" events on CI VMs and VM memory compaction etc making things freeze up

I'm able to reproduce the error in a VM on DigitalOcean (Ubuntu OS), using the same Docker command that we use in CI. In the same container, starting the php built-in server and telling Selenium to use that as the URL allows the test to pass.

Here are the request logs for npm run selenium-test:repo -- --spec repo/tests/selenium/specs/tainted-ref.js. In apache, the test fails with:

[0-0] Error in "the Tainted icon should appear and disappear correctly"
Can't call click on element with selector ".wikibase-toolbar-button-edit" because element wasn't found
[0-0] FAILED in chrome - /repo/tests/selenium/specs/tainted-ref.js

 "spec" Reporter:
------------------------------------------------------------------
[Chrome Headless 90.0.4430.93 linux #0-0] Spec: /workspace/src/extensions/Wikibase/repo/tests/selenium/specs/tainted-ref.js
[Chrome Headless 90.0.4430.93 linux #0-0] Running: Chrome Headless (v90.0.4430.93) on linux
[Chrome Headless 90.0.4430.93 linux #0-0] Session ID: efbb6e37-1a40-4785-9a32-a9a4d49f8a23
[Chrome Headless 90.0.4430.93 linux #0-0]
[Chrome Headless 90.0.4430.93 linux #0-0] the Tainted icon
[Chrome Headless 90.0.4430.93 linux #0-0]    ✖ should appear and disappear correctly
[Chrome Headless 90.0.4430.93 linux #0-0]
[Chrome Headless 90.0.4430.93 linux #0-0] 1 failing (8.1s)
[Chrome Headless 90.0.4430.93 linux #0-0]
[Chrome Headless 90.0.4430.93 linux #0-0] 1) the Tainted icon should appear and disappear correctly
[Chrome Headless 90.0.4430.93 linux #0-0] Can't call click on element with selector ".wikibase-toolbar-button-edit" because element wasn't found
[Chrome Headless 90.0.4430.93 linux #0-0] Error: Can't call click on element with selector ".wikibase-toolbar-button-edit" because element wasn't found
[Chrome Headless 90.0.4430.93 linux #0-0]     at ItemPage.clickEditOnStatementElement (/workspace/src/extensions/Wikibase/node_modules/wdio-wikibase/pagesections/main.statement.section.js:188:70)
[Chrome Headless 90.0.4430.93 linux #0-0]     at ItemPage.clickEditOnStatement (/workspace/src/extensions/Wikibase/node_modules/wdio-wikibase/pagesections/main.statement.section.js:176:8)
[Chrome Headless 90.0.4430.93 linux #0-0]     at ItemPage.editStatementValue (/workspace/src/extensions/Wikibase/node_modules/wdio-wikibase/pagesections/main.statement.section.js:200:8)
[Chrome Headless 90.0.4430.93 linux #0-0]     at Context.it (/workspace/src/extensions/Wikibase/repo/tests/selenium/specs/tainted-ref.js:49:12)

This is the screenshot

should-appear-and-disappear-correctly.png (900×1 px, 129 KB)

where you can see that the edit toolbar item is visible.

What's odd is that if I view the URL in my browser, the item is definitely present, and clickable. And if I change the line above the .click() statement to use waitForClickable(), webdriver tells me that the item is clickable.

php built-in server:

[Thu Jul  1 12:32:31 2021] 127.0.0.1:52938 [200]: //api.php?format=json
[Thu Jul  1 12:32:31 2021] 127.0.0.1:52940 [200]: //api.php?format=json
[Thu Jul  1 12:32:31 2021] 127.0.0.1:52942 [200]: //api.php?format=json
[Thu Jul  1 12:32:31 2021] 127.0.0.1:52944 [200]: //api.php?format=json
[Thu Jul  1 12:32:31 2021] 127.0.0.1:52946 [301]: //index.php?title=Special%3AEntityPage%2FQ9
[Thu Jul  1 12:32:32 2021] 127.0.0.1:52948 [200]: /index.php/Item:Q9
[Thu Jul  1 12:32:32 2021] 127.0.0.1:52950 [200]: /load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector
[Thu Jul  1 12:32:33 2021] 127.0.0.1:52952 [200]: /load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector
[Thu Jul  1 12:32:33 2021] 127.0.0.1:52968 [200]: /load.php?lang=en&modules=ext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init%7Cext.visualEditor.progressBarWidget%2CsupportCheck%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve&skin=vector&version=164jy
[Thu Jul  1 12:32:34 2021] 127.0.0.1:52970 [200]: /load.php?lang=en&modules=dataValues%2Cjquery%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%2Csite%2CvalueFormatters%2CvalueParsers%2Cvue%2Cwikibase%7CdataValues.DataValue%2CTimeValue%2Cvalues%7Cext.cx.eventlogging.campaigns%7Cext.eventLogging%2CnavigationTiming%7Cext.uls.common%2Ccompactlinks%2Ci18n%2Cinterface%2Clanguagenames%2Cmediawiki%2Cmessages%2Cpreferences%2Cwebfonts%7Cjquery.animateWithEvent%2Cclient%2Ccookie%2Ci18n%2Cinputautoexpand%2Cspinner%2CtextSelection%2Ctipsy%2Cui%2Culs%2Cvalueview%7Cjquery.event.special.eachchange%7Cjquery.spinner.styles%7Cjquery.ui.commonssuggester%2Clanguagesuggester%2Csuggester%7Cjquery.uls.data%2Cgrid%7Cjquery.util.getDirectionality%7Cjquery.valueview.Expert%2CExpertExtender%2CExpertStore%7Cjquery.valueview.experts.CommonsMediaType%2CEmptyValue%2CGeoShape%2CGlobeCoordinateInput%2CMonolingualText%2CQuantityInput%2CStringValue%2CTabularData%2CTimeInput%2CUnDeserializableValue%7Cjquery.wikibase.entityselector%2Cwbtooltip%7Cmediawiki.ForeignApi%2CString%2CTitle%2CUri%2Capi%2Cbase%2Ccldr%2Ccookie%2CjqueryMsg%2Clanguage%2Cstorage%2Cuser%2Cutil%7Cmediawiki.ForeignApi.core%7Cmediawiki.editfont.styles%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%7Cmediawiki.page.watch.ajax%7Cmediawiki.ui.button%7Cmw.config.values.wbDataBridgeConfig%2CwbDataTypes%2CwbRefTabsEnabled%2CwbRepo%2CwbSiteDetails%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui-toolbars.icons%7Coojs-ui-widgets.icons%7Coojs-ui-windows.icons%7Coojs-ui.styles.indicators%7Cskins.vector.legacy.js%7Cuser.defaults%7Cutil.ContentLanguages%2CExtendable%2CMessageProvider%2CMessageProviders%2CNotifier%2ChighlightSubstring%2Cinherit%7CvalueParsers.ValueParserStore%2Cparsers%7Cwikibase.EntityInitializer%2CSite%2CWikibaseContentLanguages%2CbuildErrorOutput%2Cdatamodel%7Cwikibase.api.RepoApi%2CValueCaller%7Cwikibase.client.data-bridge.init%7Cwikibase.entityChangers.EntityChangersFactory%7Cwikibase.entityPage.entityLoaded%7Cwikibase.experts.Entity&skin=vector&version=d3cez
[Thu Jul  1 12:32:34 2021] 127.0.0.1:52972 [200]: /load.php?lang=en&modules=wikibase.experts.Item%2CProperty%2C__namespace%2Cmodules%7Cwikibase.formatters.ApiValueFormatter%7Cwikibase.getLanguageNameByCode%2CgetUserLanguages%2Cserialization%2Csites%2Ctainted-ref%2Ctemplates%7Cwikibase.ui.entityViewInit%2Centitysearch%7Cwikibase.utilities.ClaimGuidGenerator%7Cwikibase.view.ControllerViewFactory%2CReadModeViewFactory%2C__namespace&skin=vector&version=irwsu
[Thu Jul  1 12:32:34 2021] 127.0.0.1:52974 [200]: /index.php/Special:EntityData/Q9.json?revision=28
[Thu Jul  1 12:32:34 2021] 127.0.0.1:52976 [200]: /load.php?lang=en&modules=wikibase.mediainfo.getDeserializer%7Cwikibase.mediainfo.serialization.MediaInfoDeserializer&skin=vector&version=i9esr
[Thu Jul  1 12:32:34 2021] 127.0.0.1:52978 [200]: /load.php?lang=en&modules=ext.uls.webfonts.fonts%2Crepository%7Cjquery.webfonts&skin=vector&version=gs7vw
[Thu Jul  1 12:32:35 2021] 127.0.0.1:52980 [200]: /api.php?action=wbparsevalue&format=json&parser=wikibase-entityid&values=%1FP9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:35 2021] 127.0.0.1:52982 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22ExampleValue%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fplain&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:35 2021] 127.0.0.1:52984 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22refString%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fplain&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:35 2021] 127.0.0.1:52986 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%7B%22id%22%3A%22P9%22%7D%2C%22type%22%3A%22wikibase-entityid%22%7D&generate=text%2Fplain&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:35 2021] 127.0.0.1:52988 [200]: /load.php?lang=en&modules=ext.uls.ime%7Cjquery.ime&skin=vector&version=o8jpg
[Thu Jul  1 12:32:36 2021] 127.0.0.1:52992 [200]: /load.php?lang=en&modules=ext.uls.displaysettings%2Clanguagesettings%7Cmediawiki.ui.checkbox%2Cradio&skin=vector&version=1q8jt
[Thu Jul  1 12:32:36 2021] 127.0.0.1:52994 [200]: /api.php?action=wbparsevalue&format=json&parser=string&values=%1FnewValue0.8461497974592946-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:36 2021] 127.0.0.1:52996 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22newValue0.8461497974592946-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fhtml%3B%20disposition%3Dverbose-preview&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:36 2021] 127.0.0.1:52998 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22newValue0.8461497974592946-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fplain&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:36 2021] 127.0.0.1:53000 [200]: /api.php?action=query&format=json&meta=tokens&type=csrf
[Thu Jul  1 12:32:36 2021] 127.0.0.1:53002 [200]: /api.php
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53004 [200]: /api.php?action=wbparsevalue&format=json&parser=wikibase-entityid&values=%1FP9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53006 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22refString%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fhtml%3B%20disposition%3Dverbose-preview&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53010 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22refString%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fhtml%3B%20disposition%3Dverbose-preview&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53012 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%7B%22id%22%3A%22P9%22%7D%2C%22type%22%3A%22wikibase-entityid%22%7D&generate=text%2Fhtml&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53014 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22newValue0.8461497974592946-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fplain&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53016 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22refString%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fplain&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[Thu Jul  1 12:32:37 2021] 127.0.0.1:53022 [200]: /api.php?action=wbformatvalue&format=json&datavalue=%7B%22value%22%3A%22refString%22%2C%22type%22%3A%22string%22%7D&generate=text%2Fhtml%3B%20disposition%3Dverbose-preview&property=P9&errorformat=plaintext&uselang=en&options=%7B%22lang%22%3A%22en%22%7D
[

apache:

2021-07-01T13:28:26	34418	59	POST	http://127.0.0.1:9413/api.php?format=json	application/json	-	mwbot/1.0.10	-
2021-07-01T13:28:26	153370	143	POST	http://127.0.0.1:9413/api.php?format=json	application/json	-	mwbot/1.0.10	-
2021-07-01T13:28:26	81856	59	POST	http://127.0.0.1:9413/api.php?format=json	application/json	-	mwbot/1.0.10	-
2021-07-01T13:28:26	210159	789	POST	http://127.0.0.1:9413/api.php?format=json	application/json	-	mwbot/1.0.10	-
2021-07-01T13:28:27	136402	20	GET	http://127.0.0.1:9413/index.php?title=Special%3AEntityPage%2FQ11	text/html	-	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	188591	6084	GET	http://127.0.0.1:9413/index.php?title=Item:Q11	text/html	-	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	168	22188	GET	http://127.0.0.1:9413/resources/assets/wiki.png?4d48b	image/png	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	166660	13828	GET	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	text/css	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	173	159	GET	http://127.0.0.1:9413/skins/Vector/resources/common/images/bullet-icon.svg?d4515	image/svg+xml	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	169	717	GET	http://127.0.0.1:9413/extensions/UniversalLanguageSelector/lib/jquery.uls/images/language.svg?80b0b	image/svg+xml	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	165	254	GET	http://127.0.0.1:9413/skins/Vector/resources/common/images/user-avatar.svg?b7f58	image/svg+xml	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	161	280	GET	http://127.0.0.1:9413/skins/Vector/resources/common/images/search.svg?bbf78	image/svg+xml	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	167	271	GET	http://127.0.0.1:9413/extensions/Wikibase/client/resources/images/edit.svg?34bc5	image/svg+xml	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.pt%7Cext.visualEditor.desktopArticleTarget.noscript%7Cjquery.wikibase.toolbar.styles%7Cskins.vector.styles.legacy%7Cwikibase.client.data-bridge.externalModifiers%7Cwikibase.client.init%7Cwikibase.common&only=styles&skin=vector	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	169	3839	GET	http://127.0.0.1:9413/resources/assets/poweredby_mediawiki_88x31.png	image/png	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:27	616474	19948	GET	http://127.0.0.1:9413/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector	text/javascript	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:28	163162	15520	GET	http://127.0.0.1:9413/load.php?lang=en&modules=ext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init%7Cext.visualEditor.progressBarWidget%2CsupportCheck%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve&skin=vector&version=164jy	text/javascript	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:28	243621	104809	GET	http://127.0.0.1:9413/load.php?lang=en&modules=wikibase.experts.Item%2CProperty%2C__namespace%2Cmodules%7Cwikibase.formatters.ApiValueFormatter%7Cwikibase.getLanguageNameByCode%2CgetUserLanguages%2Cserialization%2Csites%2Ctainted-ref%2Ctemplates%7Cwikibase.ui.entityViewInit%2Centitysearch%7Cwikibase.utilities.ClaimGuidGenerator%7Cwikibase.view.ControllerViewFactory%2CReadModeViewFactory%2C__namespace&skin=vector&version=irwsu	text/javascripthttp://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:28	441987	419214	GET	http://127.0.0.1:9413/load.php?lang=en&modules=dataValues%2Cjquery%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%2Csite%2CvalueFormatters%2CvalueParsers%2Cvue%2Cwikibase%7CdataValues.DataValue%2CTimeValue%2Cvalues%7Cext.cx.eventlogging.campaigns%7Cext.eventLogging%2CnavigationTiming%7Cext.uls.common%2Ccompactlinks%2Ci18n%2Cinterface%2Clanguagenames%2Cmediawiki%2Cmessages%2Cpreferences%2Cwebfonts%7Cjquery.animateWithEvent%2Cclient%2Ccookie%2Ci18n%2Cinputautoexpand%2Cspinner%2CtextSelection%2Ctipsy%2Cui%2Culs%2Cvalueview%7Cjquery.event.special.eachchange%7Cjquery.spinner.styles%7Cjquery.ui.commonssuggester%2Clanguagesuggester%2Csuggester%7Cjquery.uls.data%2Cgrid%7Cjquery.util.getDirectionality%7Cjquery.valueview.Expert%2CExpertExtender%2CExpertStore%7Cjquery.valueview.experts.CommonsMediaType%2CEmptyValue%2CGeoShape%2CGlobeCoordinateInput%2CMonolingualText%2CQuantityInput%2CStringValue%2CTabularData%2CTimeInput%2CUnDeserializableValue%7Cjquery.wikibase.entityselector%2Cwbtooltip%7Cmediawiki.ForeignApi%2CString%2CTitle%2CUri%2Capi%2Cbase%2Ccldr%2Ccookie%2CjqueryMsg%2Clanguage%2Cstorage%2Cuser%2Cutil%7Cmediawiki.ForeignApi.core%7Cmediawiki.editfont.styles%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%7Cmediawiki.page.watch.ajax%7Cmediawiki.ui.button%7Cmw.config.values.wbDataBridgeConfig%2CwbDataTypes%2CwbRefTabsEnabled%2CwbRepo%2CwbSiteDetails%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui-toolbars.icons%7Coojs-ui-widgets.icons%7Coojs-ui-windows.icons%7Coojs-ui.styles.indicators%7Cskins.vector.legacy.js%7Cuser.defaults%7Cutil.ContentLanguages%2CExtendable%2CMessageProvider%2CMessageProviders%2CNotifier%2ChighlightSubstring%2Cinherit%7CvalueParsers.ValueParserStore%2Cparsers%7Cwikibase.EntityInitializer%2CSite%2CWikibaseContentLanguages%2CbuildErrorOutput%2Cdatamodel%7Cwikibase.api.RepoApi%2CValueCaller%7Cwikibase.client.data-bridge.init%7Cwikibase.entityChangers.EntityChangersFactory%7Cwikibase.entityPage.entityLoaded%7Cwikibase.experts.Entity&skin=vector&version=rctay	text/javascript	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:28	188287	493	GET	http://127.0.0.1:9413/index.php?title=Special:EntityData/Q11.json&revision=33	application/json	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US
2021-07-01T13:28:29	103289	5556	GET	http://127.0.0.1:9413/load.php?lang=en&modules=ext.uls.webfonts.fonts%2Crepository%7Cjquery.webfonts&skin=vector&version=gs7vw	text/javascript	http://127.0.0.1:9413/index.php?title=Item:Q11	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/90.0.4430.93 Safari/537.36	en-US

Comparing index.php?title=Item:Q2 on Apache and PHP built-in server for the tainted-ref.js test failure. I see that the JS editing UI isn't loading in the Apache version for the statement:

But with PHP built-in server it is loaded:

Comparing the network requests, the only difference I see is that PHP built-in server has a final call to: /load.php?lang=en&modules=wikibase.mediainfo.getDeserializer%7Cwikibase.mediainfo.serialization.MediaInfoDeserializer&skin=vector&version=i9esr

I don't know why that doesn't load for our Quibble + Apache setup in integration/config.

Comparing the network requests, the only difference I see is that PHP built-in server has a final call to: /load.php?lang=en&modules=wikibase.mediainfo.getDeserializer%7Cwikibase.mediainfo.serialization.MediaInfoDeserializer&skin=vector&version=i9esr

If this should be happening, and is not happening, this could indeed be the reason for the editing UI not loading.

There must be some difference in setups, or some other failure somewhere?

This ends up being required from https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/f13b4df1f0c5096d6dd338c80685b7e1db97affb/repo/resources/Resources.php#130 but there is nothing special there

I wonder if the other deps are loaded?

		'wikibase.EntityInitializer' => $moduleTemplate + [
			'packageFiles' => [
				'wikibase.EntityInitializer.js',


			'dependencies' => [
				'wikibase',
				'wikibase.serialization'
			]

After debugging the Wikibase issues @Krinkle and @kostajh found this is related to T282154

kostajh added a subtask: T282154: Wikibase: Special:EntityData compression encoding issues in local dev env.Jul 2 2021, 7:22 AM

Change 701899 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@master] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/701899

In T276428#7192804, @gerritbot wrote:

Change 701899 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@master] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/701899

This fixes the Wikibase tests. The next failures are with AbuseFilter, see T285464: HTMLForms using POST might be broken if the form action is a URL containing "?title=" for that one.

Change 701899 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/701899

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.14; 2021-07-12).Jul 2 2021, 10:00 AM

Comparing https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-selenium-docker/60361/consoleFull with https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/1990/console, it looks like the Wikibase selenium tests take 2 minutes longer with apache (6 minutes instead of 4 minutes with php built-in server).

kostajh closed subtask T285464: HTMLForms using POST might be broken if the form action is a URL containing "?title=" as Resolved.Jul 2 2021, 12:28 PM

Daimona reopened subtask T285464: HTMLForms using POST might be broken if the form action is a URL containing "?title=" as Open.Jul 2 2021, 12:31 PM

Daimona removed a subtask: T285464: HTMLForms using POST might be broken if the form action is a URL containing "?title=".

Change 702938 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] selenium: Restart php7.2-fpm after modifying LocalSettings.php

https://gerrit.wikimedia.org/r/702938

Change 702904 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@master] [DNM] Test for T276428

https://gerrit.wikimedia.org/r/702904

Change 702825 had a related patch set uploaded (by Addshore; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@REL1_35] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/702825

Change 702946 had a related patch set uploaded (by Addshore; author: Kosta Harlan):

[mediawiki/extensions/Wikibase@REL1_36] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/702946

In T276428#7193773, @gerritbot wrote:

Change 702938 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] selenium: Restart php7.2-fpm after modifying LocalSettings.php

https://gerrit.wikimedia.org/r/702938

With this one, all the Selenium tests pass https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2017/consoleFull

kostajh added a parent task: T285649: Switch all Quibble Selenium and api-testing jobs to use apache.Jul 5 2021, 10:52 AM

Change 702825 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_35] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/702825

kostajh closed subtask T282154: Wikibase: Special:EntityData compression encoding issues in local dev env as Resolved.Aug 10 2021, 11:41 AM

Change 702938 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] selenium: Restart php7.2-fpm after modifying LocalSettings.php

https://gerrit.wikimedia.org/r/702938

ReleaseTaggerBot edited projects, added MW-1.37-notes (1.37.0-wmf.20; 2021-08-23); removed MW-1.37-notes (1.37.0-wmf.14; 2021-07-12).Aug 23 2021, 7:00 AM

Can someone help diagnose (and ideally fix) this error:
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2841/console

11:31:14 [0-0] RUNNING in chrome - /tests/selenium/specs/switch_views.js
11:31:16 [0-0] Error in "Switch between views: desktop to mobile"
11:31:27 element ("#footer-places-mobileview a") still not existing after 5000ms
11:31:27 [0-0] FAILED in chrome - /tests/selenium/specs/switch_views.js

Incidentally I see that a build for a different commit succeeds:
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2842/

It seems like the test may be flaky.

Change 671188 abandoned by Kosta Harlan:

[mediawiki/extensions/Wikibase@master] WIP DNM: Selenium: videos for repo & bridge using wdio-video-reporter

Reason:

Ic0785e6810dfd70

https://gerrit.wikimedia.org/r/671188

Change 702946 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_36] EntityDataRequestHandler: Remove ob_clean() call

https://gerrit.wikimedia.org/r/702946

In T276428#7313240, @dancy wrote:
Can someone help diagnose (and ideally fix) this error:
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2841/console
11:31:14 [0-0] RUNNING in chrome - /tests/selenium/specs/switch_views.js
11:31:16 [0-0] Error in "Switch between views: desktop to mobile"
11:31:27 element ("#footer-places-mobileview a") still not existing after 5000ms
11:31:27 [0-0] FAILED in chrome - /tests/selenium/specs/switch_views.js
Incidentally I see that a build for a different commit succeeds:
https://integration.wikimedia.org/ci/job/wmf-quibble-apache-selenium-php72-docker/2842/

It seems like the test may be flaky.

Marked as skipped in rEMFR29a34de17a6c: [Selenium] Skip desktop to mobile test

@hashar what else do we need to do for this task, so that we can move on to the parent task T285649: Switch all Quibble Selenium and api-testing jobs to use apache? I guess at a minimum T199393: Reproducible deadlock in User::addToDatabase() when api.php?action=createaccount is called simultaneously by several users needs to be fixed, but is there anything else?

I think that is done, we had experimental jobs running including a wmf-quibble gating job triggered from Wikibase. I have switched the Selenium Quibble jobs to use Apache and that seems to work ;)

Change 702904 abandoned by Lucas Werkmeister (WMDE):