In T219279#6086938, @holger.knust wrote:

@tstarling enwiki worked. frwiki failed. I don't have permissions to view the 332 filter rule.

Hi, this frwiki filter is throttle for renames. I've made it temporary public and added exception for this account, it should not block the script again.

In T219279#6087202, @Framawiki wrote:

In T219279#6086938, @holger.knust wrote:

@tstarling enwiki worked. frwiki failed. I don't have permissions to view the 332 filter rule.

Hi, this frwiki filter is throttle for renames. I've made it temporary public and added exception for this account, it should not block the script again.

[unrelated] @Framawiki the condition & !('Page automatiquement déplacée lors du renommage de l’utilisateur' in summary) is probably no longer needed, since filters are skipped automatically when users are being renamed

Mentioned in SAL (#wikimedia-operations) [2020-04-28T13:30:58Z] <hknust> Restarting uppercaseTitlesForUnicodeTransition.php as part of T219279 for frwiki

@Framawiki Merci beaucoup!

Mentioned in SAL (#wikimedia-operations) [2020-05-01T13:06:27Z] <hknust> holger@mwmaint1002 Starting renameInvalidUsernames.php as part of T219279

Mentioned in SAL (#wikimedia-operations) [2020-05-01T14:18:13Z] <hknust> holger@mwmaint1002 finished renameInvalidUsernames.php (fail) as part of T219279

Where is this now? Did the maintenance script actually run?

@holger.knust : Could you please answer the last comment(s)? Thanks in advance!

Gentle nudge, this really needs to be completed.

@WDoranWMF @AMooney do you have a timeline for completion of this task?

Mentioned in SAL (#wikimedia-operations) [2020-11-10T18:31:43Z] <hknust> holger mwmaint1002 Start T219279

Script execution failed with

holger@mwmaint1002:~/T219279_Unicode$ mwscript extensions/WikimediaMaintenance/renameInvalidUsernames.php loginwiki --list=userlist-2020-04-01.txt --reason="Unicode update. See T219279 for details" 
Reading from userlist-2020-04-01.txt
ɋ       Ɋ
Wikimedia\Rdbms\DBConnectionError from line 1419 of /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php: Cannot access the database: Unknown error (10.64.48.35)
#0 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(932): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#1 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(899): Wikimedia\Rdbms\LoadBalancer->getServerConnection(0, '\xC9\x8B', 4)
#2 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1045): Wikimedia\Rdbms\LoadBalancer->getConnection(-2, Array, '\xC9\x8B', 4)
#3 /srv/mediawiki/php-1.36.0-wmf.16/includes/GlobalFunctions.php(2460): Wikimedia\Rdbms\LoadBalancer->getMaintenanceConnectionRef(-2, Array, '\xC9\x8B')
#4 /srv/mediawiki/php-1.36.0-wmf.16/extensions/WikimediaMaintenance/renameInvalidUsernames.php(75): wfGetDB(-2, Array, '\xC9\x8B')
#5 /srv/mediawiki/php-1.36.0-wmf.16/extensions/WikimediaMaintenance/renameInvalidUsernames.php(50): RenameInvalidUsernames->rename('\xC9\x8A', '\xC9\x8B', NULL)
#6 /srv/mediawiki/php-1.36.0-wmf.16/maintenance/doMaintenance.php(106): RenameInvalidUsernames->execute()
#7 /srv/mediawiki/php-1.36.0-wmf.16/extensions/WikimediaMaintenance/renameInvalidUsernames.php(182): require_once('/srv/mediawiki/...')
#8 /srv/mediawiki/multiversion/MWScript.php(101): require_once('/srv/mediawiki/...')
#9 {main}

Mentioned in SAL (#wikimedia-operations) [2020-11-10T19:06:26Z] <hknust> holger mwmaint1002 Stop T219279

pc2010 seems to be lagging behind. This is a non-issue for production, given it is codfw, but noting it here because it may alert if the writing trends continue. This maintenance is the most likely explanation- as it started at the exact time the log indicates (but I am not 100% sure about it).

From looking at the code, it seems like the user list ought to have three fields, the first one being the name of the wiki. That appears to be missing. Someone can correct me on that later if they know better.

Was this ever finished? This has come up again on https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Should_I_have_been_able_to_create_these_lowercase_Cyrillic_redirects? where someone was able to create a lowercase page because the letter is still in wmf-config/Php72ToUpper.php.

No, I believe the final step post-upgrade hasn't yet been done.

Production configuration still runs with the "HHVM-like" shim applied, suggesting this migration indeed has not been completed.

# Part of the HHVM => PHP7.2 migration. Adds an array of unicode chars
# that have broken uppercasing in HHVM. In this phase, we want php7 to behave
# like HHVM. See T219279 for details.
$wgOverrideUcfirstCharacters = include __DIR__ . '/Php72ToUpper.php';

Gosh this it's hard to parse what's going on here and the folks most involved are not working here anymore. I've tried to parse what exactly was and wasn't done from the logs, so I'll assume the the checkboxes in T219279#5362460 are more-or-less correct.

• Determine the schedule to do these next steps.
• Generate a list of affected titles, and announce the upcoming change on User-notice.
• Run uppercaseTitlesForUnicodeTransition.php and renameInvalidUsernames.php on all wikis. My current plan would be to use --suffix ' (former Unicode lowercase)' for handling of collisions.
• Reverse https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/505487 so HHVM uses PHP 7.2's mappings. Remove the override from https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/505487.
• Run uppercaseTitlesForUnicodeTransition.php and renameInvalidUsernames.php on all wikis again, in case someone created new problem titles while the scripts were running the first time.

I'll first dry-run the uppercaseTitlesForUnicodeTransition.php, worst case schenario I just run the script that's already been run before.

I'll first dry-run the uppercaseTitlesForUnicodeTransition.php, worst case schenario I just run the script that's already been run before.

So, I've dry-run the uppercaseTitlesForUnicodeTransition.php with current Php72ToUpper.php overrides map in production and it reports nothing to rename, which means someone has probably already done it. Unfortunately, I think that's actually not what needs to be done here.

Currently, we're running php7.2 in production and we have strtoupper overrides enabled, forcing it to behave just like HHVM ( for example, ƀ is still uppercased as ƀ ). When we remove the overrides, we will start using native uppercasing ( for example ƀ will be uppercased as Ƀ ). So if there are any pages that start with HHVM-not-uppercased letters, these pages need to be renamed to uppercased letters before we can remove the overrides.

I've generated a new 'fake' overrides map, from current MW uppercasing to PHP 7.2 native uppercasing here

So, first I've dry-run the script

foreachwiki uppercaseTitlesForUnicodeTransition.php --charmap current_to_php7_overrides.php --userlist /tmp/rename_users_for_uppercase.txt --suffix ' (former Unicode lowercase)' > results.txt

to find out there's 809 pages overall that need renaming and quite some usernames. First problem is that the uppercaseTitlesForUnicodeTransition.php script generates a list of usernames for us, but in a format that's not compatible with renameInvalidUsernames.php. Let's make them compatible first.

Change 726083 had a related patch set uploaded (by Ppchelko; author: Ppchelko):

[mediawiki/core@master] uppercaseTitlesForUnicodeTransition: improve userlist format

https://gerrit.wikimedia.org/r/726083

Change 726083 merged by jenkins-bot:

[mediawiki/core@master] uppercaseTitlesForUnicodeTransition: improve userlist format

https://gerrit.wikimedia.org/r/726083

Change 726589 had a related patch set uploaded (by Ppchelko; author: Ppchelko):

[mediawiki/core@wmf/1.38.0-wmf.2] uppercaseTitlesForUnicodeTransition: improve userlist format

https://gerrit.wikimedia.org/r/726589

Change 726589 merged by jenkins-bot:

[mediawiki/core@wmf/1.38.0-wmf.2] uppercaseTitlesForUnicodeTransition: improve userlist format

https://gerrit.wikimedia.org/r/726589

Mentioned in SAL (#wikimedia-operations) [2021-10-05T13:23:54Z] <ppchelko@deploy1002> Synchronized php-1.38.0-wmf.2/maintenance/uppercaseTitlesForUnicodeTransition.php: Backport uppercaseTitlesForUnicodeTransition.php maintenance script improvements T219279 (duration: 00m 58s)

Change 726623 had a related patch set uploaded (by Ppchelko; author: Ppchelko):

[operations/mediawiki-config@master] Remove mb_strtoupper overrides for HHVM

https://gerrit.wikimedia.org/r/726623

Mentioned in SAL (#wikimedia-operations) [2021-10-05T13:46:27Z] <Pchelolo> run renameInvalidUsernames.php --wiki loginwiki --list /tmp/rename_users_for_uppercase_all.txt T219279

Renaming users completed. There's 4 accounts across all sites that we can't capitalize because the capitalized account exists. Neither of those 4 accounts appear to be active editors, nothing can be done here really, the 4 accounts will become unreachable.

Mentioned in SAL (#wikimedia-operations) [2021-10-05T14:30:07Z] <Pchelolo> run foreachwiki uppercaseTitlesForUnicodeTransition.php --charmap current_to_php7_overrides.php T219279

Script run finished. We've has some AbuseFilter violations:

• https://cs.wikipedia.org/wiki/Speciáln%C3%AD:Filtry_zneužit%C3%AD/14
• https://ru.wikipedia.org/wiki/Служебная:Фильтр_злоупотреблений/16
• https://ru.wikipedia.org/wiki/Служебная:Фильтр_злоупотреблений/11

If someone could be so kind to disable these filters, or exclude Maintenance script user from them, I'll rerun the renaming script for these wikis.

In any case it seems like the pages we've failed to rename are just redirects, so we can proceed with removing the overrides from production, e.g. https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/726623

I'll rerun the cleanup script again after it's deployed, just for good measure.

In T219279#7402408, @Pchelolo wrote:

Renaming users completed. There's 4 accounts across all sites that we can't capitalize because the capitalized account exists. Neither of those 4 accounts appear to be active editors, nothing can be done here really, the 4 accounts will become unreachable.

Should they be renamed to something else, like was done with SUL finalization?

Should they be renamed to something else, like was done with SUL finalization?

Yeah, can do. Like '<capitalized letter>_T219279'?

Was just about to ask this. While rare, there are cases of accounts making their first edits years after creation. Maybe systematically rename them to something like <capitalized username> (technical rename), with usertalk notes pointing them to Special:GlobalRenameRequest?

Edit: Whoops, didn't get the notification for your post till after I'd posted this, Pchelolo.

with usertalk notes pointing them to Special:GlobalRenameRequest?

That's the issue, I can only leave a note on User_Talk:<capitalized username> (technical rename) page, not User_Talk:<unreachable_lowercase_username> since that user_talk will also be unreachable. And the capitalized version belongs to a different user. And without a notice if the owner of the account decides to come back, how would they know where to look for the note..

In T219279#7403003, @Pchelolo wrote:

If someone could be so kind to disable these filters, or exclude Maintenance script user from them, I'll rerun the renaming script for these wikis.

This is a regression in AbuseFilter, see T212082#4826766. Years ago I had refactored MovePage to specifically allow exempting maintenance scripts and server-initiated page renames from AbuseFilters and other anti-spam checks.

In T219279#7403097, @Pchelolo wrote:

with usertalk notes pointing them to Special:GlobalRenameRequest?

That's the issue, I can only leave a note on User_Talk:<capitalized username> (technical rename) page, not User_Talk:<unreachable_lowercase_username> since that user_talk will also be unreachable. And the capitalized version belongs to a different user. And without a notice if the owner of the account decides to come back, how would they know where to look for the note..

If you globally rename the users now, and leave the notes on their home-wiki talkpages, won't they still get the "You have new messages" banner (etc.), same as any other rename?

In T219279#7403110, @Tamzin wrote:

In T219279#7403097, @Pchelolo wrote:

with usertalk notes pointing them to Special:GlobalRenameRequest?

That's the issue, I can only leave a note on User_Talk:<capitalized username> (technical rename) page, not User_Talk:<unreachable_lowercase_username> since that user_talk will also be unreachable. And the capitalized version belongs to a different user. And without a notice if the owner of the account decides to come back, how would they know where to look for the note..

If you globally rename the users now, and leave the notes on their home-wiki talkpages, won't they still get the "You have new messages" banner (etc.), same as any other rename?

Renaming users logs them out, and by default, no rename issues (email) notifications (web notifications are inaccessible -- you'd have to log in first, and for that, you have to know about a rename). Only approved queue renames do so (and AFAIK an email is mandatory for them). If I rename someone manually (for technical reasons, or because of a request on a community request page), they only notice it after they log in. Kinda easy to spot if you expect the rename, but very unlikely if you don't (and aren't particulary aware of how MW works).

In another words, there's very little chance to contact them in any way, I'm afraid.

Ah. That's a shame. You'd think after two renames I'd know that. Oh well. If anyone does try to awaken a dormant account, they'll probably just think they forgot the password.

Change 726623 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove mb_strtoupper overrides for HHVM

https://gerrit.wikimedia.org/r/726623

Mentioned in SAL (#wikimedia-operations) [2021-10-05T18:04:24Z] <ppchelko@deploy1002> Synchronized wmf-config/CommonSettings.php: Remove mb_strtoupper overrides for HHVM T219279 CS.php (duration: 01m 06s)

Mentioned in SAL (#wikimedia-operations) [2021-10-05T18:11:06Z] <ppchelko@deploy1002> Synchronized wmf-config: Remove mb_strtoupper overrides for HHVM T219279 Php72ToUpper.php removal (duration: 01m 06s)

Ok, we're done with removal of overrides. Now production uppercasing is just regular PHP 7.2 uppercasing.

For example, https://test.wikipedia.org/wiki/ɋ is not https://test.wikipedia.org/wiki/Ɋ and so on.

Just to be completely sure nothing was left behind I'll run the script one more time in case some pages were re-created from the last script run, and we're done here.

Change 726672 had a related patch set uploaded (by Ppchelko; author: Ppchelko):

[mediawiki/core@master] Remove phpCharMappings that are no longer nesessary

https://gerrit.wikimedia.org/r/726672

Change 726672 merged by jenkins-bot:

[mediawiki/core@master] mediawiki.Title: Regenerate phpCharMappings against plain PHP 7.2

https://gerrit.wikimedia.org/r/726672

@Pchelolo Ping? Can you please review the renames that failed? See https://meta.wikimedia.org/wiki/Special:GlobalRenameProgress for a list of them.

This task (or possibly a prior similar one) was noticed at https://en.wikipedia.org/wiki/WP:VPT#Automatically_renamed_users , just check in to see whether the behavior of Special:Contribs is expected.

https://en.wikipedia.org/wiki/WP:VPT#Automatically_renamed_users

Hello! See User:Ʝ and User:DZoo — two users who seem to have disappeared during a rename from lowercase to capital letters. Their user pages' history show them being created by users with the lowercase name, but clicking their user link or contribs link brings you to the nonexistent username with the capital first character.

In T219279#7488442, @TheDJ wrote:

https://en.wikipedia.org/wiki/WP:VPT#Automatically_renamed_users

Hello! See User:Ʝ and User:DZoo — two users who seem to have disappeared during a rename from lowercase to capital letters. Their user pages' history show them being created by users with the lowercase name, but clicking their user link or contribs link brings you to the nonexistent username with the capital first character.

This is probably a problem with the user rename procedure.

What I see in the database is that the revision that created the page for user DZoo is tied to user Dzoo in the enwiki database, which seems to be still present in the user table (id 7026957)

(I was the one at the Village Pump.) It looks like the renames didn't entirely work — has this happened with other users?

Mentioned in SAL (#wikimedia-operations) [2022-06-27T09:01:11Z] <taavi> mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=enwiki --logwiki=metawiki 'Dzoo' 'DZoo' # fixing stuck rename on T219279