Move the Query Builder repository to Gerrit
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Michael
	Mar 10 2021, 4:31 PM

Description

Things that need to be done to consider the move to Gerrit complete:

create Query Builder Gerrit repo -> https://wikitech.wikimedia.org/wiki/Gerrit#Creating_new_repositories
- https://gerrit.wikimedia.org/r/admin/repos/wikidata/query-builder
somehow figure out Cypress in CI (see T230729 for some prior work) and branch hosting, if needed
- have a look at https://github.com/wikimedia/integration-config/blob/master/dockerfiles/node10-test-browser/run-with-xvfb.sh
- waiting on the server to be ready for cypress: https://docs.cypress.io/guides/continuous-integration/introduction.html#Boot-your-server

create Jenkins test-job, it needs docker with nodejs? -> query service ui
~~[ ] create Jenkins merge job?~~
adjust translate-wiki to add their translations to the Gerrit repo
create mirror to https://github.com/wikimedia/wikidata-query-builder/
archive repo at https://github.com/wmde/query-builder/

manually fix test-system to pull from Gerrit/gittiles instead of GitHub (though it could pull from the mirror)
- Alternatively: can Jenkins push after merge to toolforge and trigger the build script there? That would then be real CD for our CI... 🥺

Details

Subject	Repo	Branch	Lines +/-
Replace links to github for source and license	wikidata/query-builder	master	+5 -5
Introduce query-builder job so it can use npm 6.14.* instead	integration/config	master	+22 -2
Wikidata: Use the new address of query-builder repo	translatewiki	master	+3 -3
Add configuration for new wikidata/query-builder repo	integration/config	master	+4 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• amy_rc	T277060 Move the Query Builder repository to Gerrit
		Resolved		None	T272130 Consider moving the Wikidata Query Builder repository from github to gerrit

Event Timeline

Michael created this task.Mar 10 2021, 4:31 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 10 2021, 4:31 PM

Michael added a subtask: T272130: Consider moving the Wikidata Query Builder repository from github to gerrit.Mar 10 2021, 4:32 PM

Michael updated the task description. (Show Details)Mar 10 2021, 4:39 PM

Michael updated the task description. (Show Details)Mar 10 2021, 4:45 PM

Addshore subscribed.Mar 10 2021, 5:59 PM

Michael updated the task description. (Show Details)Mar 11 2021, 9:40 AM

Michael updated the task description. (Show Details)Mar 11 2021, 10:18 AM

Change 670898 had a related patch set uploaded (by Michael Große; owner: Michael Große):
[integration/config@master] Add configuration for new wikidata/query-builder repo

https://gerrit.wikimedia.org/r/670898

gerritbot added a project: Patch-For-Review.Mar 11 2021, 5:49 PM

Ladsgroup closed subtask T272130: Consider moving the Wikidata Query Builder repository from github to gerrit as Resolved.Mar 15 2021, 6:21 AM

Ladsgroup mentioned this in T272130: Consider moving the Wikidata Query Builder repository from github to gerrit.

Ladsgroup updated the task description. (Show Details)

Michael updated the task description. (Show Details)Mar 15 2021, 9:36 AM

Change 670898 merged by jenkins-bot:
[integration/config@master] Add configuration for new wikidata/query-builder repo

https://gerrit.wikimedia.org/r/670898

Maintenance_bot removed a project: Patch-For-Review.Mar 16 2021, 11:10 AM

Change 673183 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[integration/config@master] Introduce query-builder job so it can use npm 6.14.* instead

https://gerrit.wikimedia.org/r/673183

gerritbot added a project: Patch-For-Review.Mar 18 2021, 5:07 AM

Change 673183 merged by jenkins-bot:
[integration/config@master] Introduce query-builder job so it can use npm 6.14.* instead

https://gerrit.wikimedia.org/r/673183

Mentioned in SAL (#wikimedia-releng) [2021-03-18T15:55:21Z] <addshore> reload zuul for Introduce query-builder job so it can use npm 6.14.* instead [integration/config] - https://gerrit.wikimedia.org/r/673183 T277060

Maintenance_bot removed a project: Patch-For-Review.Mar 18 2021, 4:10 PM

Cypress currently fails with the following error:

00:04:24.868 > CYPRESS_API_URL=${VUE_APP_WIKIBASE_API_URL} CYPRESS_BASE_URL=${CYPRESS_BASE_URL:-http://localhost:8080} cypress run --browser chromium --headless
00:04:24.868 
00:04:24.868 It looks like this is your first time using Cypress: 6.3.0
00:04:24.868 
00:04:24.868 [16:38:24]  Verifying Cypress can run /cache/Cypress/6.3.0/Cypress [started]

00:04:26.669 [16:38:26]  Verifying Cypress can run /cache/Cypress/6.3.0/Cypress [failed]
00:04:26.670 Cypress failed to start.
00:04:26.670 
00:04:26.670 This is usually caused by a missing library or dependency.
00:04:26.670 
00:04:26.670 The error below should indicate which dependency is missing.
00:04:26.670 
00:04:26.670 https://on.cypress.io/required-dependencies
00:04:26.670 
00:04:26.670 If you are using Docker, we provide containers with all required dependencies installed.
00:04:26.670 
00:04:26.670 ----------
00:04:26.670 
00:04:26.670 Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
00:04:26.670 [744:0318/163825.667206:ERROR:browser_main_loop.cc(1434)] Unable to open X display.
00:04:26.670 Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
00:04:26.670 [758:0318/163826.663749:ERROR:browser_main_loop.cc(1434)] Unable to open X display.
00:04:26.670 
00:04:26.670 undefined:0
00:04:26.670 
00:04:26.670 
00:04:26.670 illegal access
00:04:26.670 (Use `Cypress --trace-uncaught ...` to show where the exception was thrown)
00:04:26.670 
00:04:26.670 ----------
00:04:26.670 
00:04:26.670 Platform: linux (Debian - 9.12)
00:04:26.670 Cypress Version: 6.3.0

I added a cypress info step before that to get maybe more insights and that is the output:

00:02:46.323 > @wmde/query-builder@1.0.0 test:cypress-info /src
00:02:46.323 > cypress info
00:02:46.323 
00:02:48.342 
00:02:48.342 Proxy Settings: none detected
00:02:48.345 Environment Variables: none detected
00:02:48.345 
00:02:48.346 Application Data: /tmp/xdg-config-home/cypress/cy/development
00:02:48.346 Browser Profiles: /tmp/xdg-config-home/cypress/cy/development/browsers
00:02:48.346 Binary Caches: /cache/Cypress
00:02:48.347 
00:02:48.366 Cypress Version: 6.3.0
00:02:48.366 System Platform: linux (Debian - 9.12)
00:02:48.366 System Memory: 25.3 GB free 8.8 GB
00:02:49.109

Notably, and that is different from the result I get locally, this does not mention any available browsers whatsoever.

Are there maybe no browsers installed or installed in a way that is not legible to Cypress?

I tried to find more information about the docker container that we are using, but got stuck at https://docker-registry.wikimedia.org/releng/node10-test-browser/tags/
I couldn't figure out where to find more information about those specific tags. We're currently using 0.6.3. How is that different from 0.6.3-s1 and 0.6.3-s2? Where is the source for those images?

I tried to find more information about the docker container that we are using, but got stuck at https://docker-registry.wikimedia.org/releng/node10-test-browser/tags/
I couldn't figure out where to find more information about those specific tags. We're currently using 0.6.3. How is that different from 0.6.3-s1 and 0.6.3-s2?

You can find the information on the changes in versions of the images in their respective changelog files, e.g. https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/dockerfiles/node10-test-browser/changelog
If I do not get it wrong, -s2, -s3 versions are versions of the image where the image itself did not change but the base image(s) were updated/refreshed.

Where is the source for those images?

Dockerfiles are in the operations/config Gerrit repo. E.g. the one you mention would be https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/dockerfiles/node10-test-browser/

Ladsgroup subscribed.Mar 19 2021, 1:29 AM

Seemingly the browser both exist at /usr/bin/chromium and /usr/local/bin/firefox respectively.

These paths are also in the PATH

nobody@a2cf77adb126:/src$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

and chromium via which

docker run --rm -it --entrypoint bash docker-registry.wikimedia.org/releng/node10-test-browser:0.6.3-s2
nobody@284a082ad805:/src$ which chromium
/usr/bin/chromium

https://docs.cypress.io/guides/references/troubleshooting.html#Launching-browsers
suggests using debug logging for more output

Another way to log what is found by Cypress is to run Cypress with the DEBUG environment variable set to cypress:launcher. This will print information about the found browsers and their properties to the terminal.

https://docs.cypress.io/guides/references/troubleshooting.html#Print-DEBUG-logs

Thanks! https://integration.wikimedia.org/ci/job/query-builder-node10-browser-docker/5/consoleFull

Michael updated the task description. (Show Details)Mar 19 2021, 8:57 AM

Michael updated the task description. (Show Details)Mar 19 2021, 5:50 PM

Change 673552 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[translatewiki@master] Wikidata: Use the new address of query-builder repo

https://gerrit.wikimedia.org/r/673552

gerritbot added a project: Patch-For-Review.Mar 19 2021, 5:56 PM

Change 673552 merged by jenkins-bot:
[translatewiki@master] Wikidata: Use the new address of query-builder repo

https://gerrit.wikimedia.org/r/673552

jenkins-bot mentioned this in rGTWN8def2550499c: Wikidata: Use the new address of query-builder repo.Mar 19 2021, 7:20 PM

Maintenance_bot removed a project: Patch-For-Review.Mar 19 2021, 8:11 PM

Michael updated the task description. (Show Details)Mar 29 2021, 3:50 PM

Ladsgroup updated the task description. (Show Details)Mar 30 2021, 12:39 PM

Mentioned in SAL (#wikimedia-operations) [2021-03-30T12:39:22Z] <Amir1> ssh -p 29418 gerrit.wikimedia.org replication start wikidata/query-builder --wait (T277060)

This is done I assume.

Change 675809 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):
[wikidata/query-builder@master] Replace links to github for source and license

https://gerrit.wikimedia.org/r/675809

gerritbot added a project: Patch-For-Review.Mar 30 2021, 2:28 PM

Change 675809 merged by jenkins-bot:
[wikidata/query-builder@master] Replace links to github for source and license

https://gerrit.wikimedia.org/r/675809

Maintenance_bot removed a project: Patch-For-Review.Mar 30 2021, 4:10 PM

• amy_rc moved this task from Test (Verification) to Done on the Wikidata Query Builder board.Mar 31 2021, 9:57 AM