Page MenuHomePhabricator

wdio-cucumber-framework fails on NodeJS due to fibers@2.x under nodejs10
Closed, ResolvedPublic

Description

INFO:zuul.CloneMapper:Expansion completed.
INFO:quibble.commands:Running webdriver test in /workspace/src/extensions/CirrusSearch

> fibers@2.0.2 install /workspace/src/extensions/CirrusSearch/node_modules/fibers
> node build.js || nodejs build.js

gyp ERR! configure error 
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (/srv/npm/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (/srv/npm/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack     at F (/srv/npm/node_modules/which/which.js:68:16)
gyp ERR! stack     at E (/srv/npm/node_modules/which/which.js:80:29)
gyp ERR! stack     at /srv/npm/node_modules/which/which.js:89:16
gyp ERR! stack     at /srv/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack     at /srv/npm/node_modules/isexe/mode.js:8:5
gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:154:21)
gyp ERR! System Linux 4.9.0-0.bpo.9-amd64
gyp ERR! command "/usr/bin/node" "/srv/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
gyp ERR! cwd /workspace/src/extensions/CirrusSearch/node_modules/fibers
gyp ERR! node -v v10.15.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
node-gyp exited with code: 1
Please make sure you are using a supported platform and node version. If you
would like to compile fibers on this machine please make sure you have setup your
build environment--
Windows + OS X instructions here: https://github.com/nodejs/node-gyp
Ubuntu users please run: `sudo apt-get install g++ build-essential`
Alpine users please run: `sudo apk add python make g++`
gyp ERR! configure error 
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (/srv/npm/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (/srv/npm/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack     at F (/srv/npm/node_modules/which/which.js:68:16)
gyp ERR! stack     at E (/srv/npm/node_modules/which/which.js:80:29)
gyp ERR! stack     at /srv/npm/node_modules/which/which.js:89:16
gyp ERR! stack     at /srv/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack     at /srv/npm/node_modules/isexe/mode.js:8:5
gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:154:21)
gyp ERR! System Linux 4.9.0-0.bpo.9-amd64
gyp ERR! command "/usr/bin/node" "/srv/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
gyp ERR! cwd /workspace/src/extensions/CirrusSearch/node_modules/fibers
gyp ERR! node -v v10.15.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
node-gyp exited with code: 1

Related Objects

Event Timeline

Somehow releng/quibble-stretch-hhvm lacks python :-(

$ docker run --rm -it --entrypoint=python docker-registry.discovery.wmnet/releng/quibble-stretch-hhvm:0.0.34
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"python\": executable file not found in $PATH": unknown.

Same for 0.0.32-1-s2

The wmf-quibble-core-vendor-mysql-hhvm-docker job has several repositories containing selenium-test and npm install passes just fine on MinervaNeue:

INFO:quibble.commands:Running webdriver test in /workspace/src/skins/MinervaNeue

> fibers@3.1.1 install /workspace/src/skins/MinervaNeue/node_modules/fibers
> node build.js || nodejs build.js

`linux-x64-64-glibc` exists; testing
Binary is fine; exiting

> fibers@3.1.1 install /workspace/src/skins/MinervaNeue/node_modules/wdio-mocha-framework/node_modules/fibers
> node build.js || nodejs build.js

`linux-x64-64-glibc` exists; testing
Binary is fine; exiting

But fails on CirrusSearch:

INFO:quibble.commands:Running webdriver test in /workspace/src/extensions/CirrusSearch

> fibers@2.0.2 install /workspace/src/extensions/CirrusSearch/node_modules/fibers
> node build.js || nodejs build.js

gyp ERR! configure error 
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (/srv/npm/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (/srv/npm/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack     at F (/srv/npm/node_modules/which/which.js:68:16)
gyp ERR! stack     at E (/srv/npm/node_modules/which/which.js:80:29)
gyp ERR! stack     at /srv/npm/node_modules/which/which.js:89:16
gyp ERR! stack     at /srv/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack     at /srv/npm/node_modules/isexe/mode.js:8:5
gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:154:21)
gyp ERR! System Linux 4.9.0-0.bpo.9-amd64
gyp ERR! command "/usr/bin/node" "/srv/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
gyp ERR! cwd /workspace/src/extensions/CirrusSearch/node_modules/fibers
gyp ERR! node -v v10.15.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
node-gyp exited with code: 1

Due to fibers@2.0.2 which is not binary compatible with NodeJS 10 :-]

I don't know how fibers@3.1.1 works without python but maybe it does not need it or ship a binary.

CirrusSearch has: wdio-cucumber-framework@1.0.1 > wdio-sync@0.7.0 > fibers@2.0.2

So I guess we need to bump wdio-cucumber-framework just like we did for wdio-mocha-framework eg b9b068e25a40b57630163dd2e56c3d2a551aa232

$ grep wdio-cucumber-framework {extensions,skins}/*/package.json
extensions/CirrusSearch/package.json:		"wdio-cucumber-framework": "1.0.1",
extensions/MobileFrontend/package.json:    "wdio-cucumber-framework": "1.1.1",
skins/MinervaNeue/package.json:		"wdio-cucumber-framework": "2.2.8",

Change 525612 had a related patch set uploaded (by Hashar; owner: Jforrester):
[mediawiki/extensions/CirrusSearch@master] build: Upgrade wdio-cucumber-framework to one that works

https://gerrit.wikimedia.org/r/525612

Change 525614 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/MobileFrontend@master] build: Upgrade wdio-cucumber-framework to one that works

https://gerrit.wikimedia.org/r/525614

hashar renamed this task from Quibble 0.0.34: node-gyp fails for extension due to lack of python to wdio-cucumber-framework fails on NodeJS due to fibers@2.x under nodejs10.Jul 25 2019, 6:11 PM

Change 525614 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] build: Upgrade wdio-cucumber-framework to one that works

https://gerrit.wikimedia.org/r/525614

Change 525632 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/MobileFrontend@wmf/1.34.0-wmf.15] build: Upgrade wdio-cucumber-framework to one that works

https://gerrit.wikimedia.org/r/525632

Change 525632 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@wmf/1.34.0-wmf.15] build: Upgrade wdio-cucumber-framework to one that works

https://gerrit.wikimedia.org/r/525632

Change 525647 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/CirrusSearch@master] build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525647

Change 525653 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/CirrusSearch@wmf/1.34.0-wmf.14] build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525653

Change 525653 abandoned by Hashar:
build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525653

Change 525654 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/CirrusSearch@wmf/1.34.0-wmf.15] build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525654

Fixed for #MobileFronted

For CirrusSearch we would first need to upgrade Cindy the Browser bot to use nodejs 10 which means upgrading it in mediawiki/vagrant ( T217113 ).

Change 525647 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525647

Change 525654 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@wmf/1.34.0-wmf.15] build: temporarily rename 'selenium-test'

https://gerrit.wikimedia.org/r/525654

We still need to restore 'selenium-test' for CirrusSearch (https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CirrusSearch/+/525612/)

That requires Cindy the Browser bot to use NodeJS 10.

Cindy the browser bot uses mediawiki/vagrant which apparently has the NodeJS 6 from stretch-wikimedia.

So I guess we will first need to upgrade NodeJS in mediawiki/vagrant to NodeJS 10 which would fix T217113 ""MediaWiki-Vagrant should use the same Node.js version as Wikimedia production"" ;-]

hashar changed the task status from Open to Stalled.Aug 19 2019, 8:29 AM
hashar edited projects, added Release-Engineering-Team-TODO; removed Quibble.

Stall / pending NodeJS being upgraded in mediawiki/vagrant T217113

Change 537149 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/CirrusSearch@master] Revert "build: temporarily rename 'selenium-test'"

https://gerrit.wikimedia.org/r/537149

Change 525612 abandoned by EBernhardson:
build: Upgrade wdio-cucumber-framework to one that works

Reason:
superseded, per referenced patch

https://gerrit.wikimedia.org/r/525612

Change 537149 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Selenium: fix selenium-daily-beta-CirrusSearch Jenkins job

https://gerrit.wikimedia.org/r/537149

Change 537369 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/CirrusSearch@master] Reenable selenium-test in CI

https://gerrit.wikimedia.org/r/537369

Change 537369 abandoned by Hashar:
Reenable selenium-test in CI

https://gerrit.wikimedia.org/r/537369

hashar claimed this task.

That was resolved by upgrading the wdio packages to a version that works on whatever nodejs.

Note fibers does not work on Node JS 16 anymore and webdriver.io tests should be moved to sync mode: T256626

This task was left open due to the sub task T217113 which is for MediaWiki Vagrant. It apparently still default to NodeJS 6 or maybe NodeJS 10 from Debian Buster. Given MediaWiki Vagrant is no more supported, I am willing to ignore as a sub task and thus mark this one resolved.