This page contains information about common issues you may encounter while working with Toolforge.
Permission denied when running a script
Make sure the script is running from your Tool Account, rather than your user account.
User preferences can't be accessed in the replicas
User preferences are considered private information and can't be accessed in the replicas.
Error message: libgcc_s.so.1 must be installed for pthread_cancel to work
If you encounter this while trying to run a job on the grid engine, the job needs more memory.
Error message: No such tool
You receive the following error message after creating a new Tool Account:
$ become <TOOL NAME>
become: no such tool '<TOOL NAME>'
- Wait a few minutes for the Tool Account creation to complete.
- Check that the spelling of the tool name is correct.
Error message: You are not a member of the group tools
You receive the following error after joining a Tool Account:
$ become <TOOL NAME>
You are not a member of the group tools.<TOOL NAME>.Any existing member of the tool's group can add you to that.
If you are already logged in via SSH when you created the a new tool, log out, and log in again to activate your new permissions.
Error message: Trying to ssh into login.toolforge.org: getting a "firstname.lastname@example.org: Permission denied (publickey,hostbased)".
If you are running ssh from the command line, make sure that the command is ssh email@example.com, where "username" is your username (and not the name of the tool you're attempting to log into).
Error message: Warning: the ECDSA host key for 'login.toolforge.org' differs from the key for the IP address '126.96.36.199'
This means the key has been updated. Try refreshing it with :ssh-keygen -R <ip>
public_html does not work in home directory
Users cannot have a public_html folder to themselves. The only web-accessible directories are under /data/project/<toolname>/public_html/.
To have a URL such as http://<username>.toolforge.org/, you must create a tool called <username>, which will create a folder called /data/project/<username>/public_html/.
Nobody--except you, the user--will *ever* be given access to your home or its files. Allowing public services to run from a home directory means that their management could not be shared or taken over if they end up abandoned, defeating the purpose.
Can't detect Cloud VPS or the project
Every Cloud VPS instance contains the file: /etc/wmflabs-project. The contents of the of the file will indicate whether the project is "tools" for the Toolforge, or "toolsbeta" for the experimental Toolforge.
You might not be the only one missing that package. Please submit a ticket in Phabricator
and ask the admins to install it project-wide. If they have reasons not to do so you can always install software locally / just for yourself.
Note that for python packages, we recommend that you use a virtualenv
local to your tool.
Only request a global package if it is difficult to install via pip (numpy, scipy, etc). For example to install the python package internetarchive run
echo "source $HOME/env/bin/activate" >> .bashrc
pip install internetarchive
Your connection may be slower depending on your location.
If you are connecting from Europe, try using Mosh
. To connect, use mosh -a login.toolforge.org
(-a to force predictive echo) (instead of ssh login.tooforge.org
Developers working in Toolforge do not have to create or set up virtual machines (i.e., Cloud VPS "instances"), because the Toolforge project admins create and manage them. The term may appear in global documentation on Wikitech but is not applicable to Toolforge.
When your connection is fine but the Toolforge bastion (login.toolforge.org, tools-sgebastion-07) is slow or unresponsive, it is likely due to NFS lag. All users of the Toolforge bastion share the same limited connection to NFS
. Tasks such as downloading large files or doing processing directly on the bastion host create a high volume of traffic that saturates the connection to NFS. Interactive tools like your shell, cd
, and ls
must then wait longer to send and receive their own traffic.
To prevent NFS lag, do not run any tasks that require heavy disk IO directly from the bastion. Instead, run them from the Grid
or inside a webservice shell
. Interactive jobs that require disk IO can also be carried out on the development bastion
(dev.toolforge.org), but they may still impact others there.
Toolforge administrators will kill processes running from the bastion that are causing excessive NFS lag.
Report issues with Toolforge Phabricator
is used for issue and work tracking across Wikimedia technical projects.
All issues in Phabricator are available for the public to view. However, to use Phabricator to report issues, you must create a developer account
When reporting issues, remember
- Phabricator is relies on volunteers. It may take some time before your issue is addressed.
- Make an effort to define the problem thoroughly and specifically.
- Where did you encounter the problem?
- What isn't working?
- How do you think it should be working.
Reporting an issue with a specific Tool
1) Find maintainers to notify and a Phabricator project to associate.
As an example, you can find details for the admin Tool (which powers the default page for admin.toolforge.org
) at the toolsadmin page for it. Any tool should provide information at a similar URL: ''https://toolsadmin.wikimedia.org/tools/id/<tool name>''
2) Create a task in Phabricator.
There are general Wikimedia documents on the needed instructions
and recommended content
for reporting issues.
Communication and support
We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.
Communicate with us
Last edited on 27 August 2021, at 22:00
Content is available under CC BY-SA 3.0
unless otherwise noted.