Future Hatstall version has been deployed two months ago, so we could have another discussion about this (but maybe it's already enabled? Should likely check on an identity defined by activity on a mailing list or Gerrit where email addresses get indexed).

In T107254#9710082, @hashar wrote:

As for CLOSED, I guess we did that to prevent further comments and/or to indicate the task resolution got verified. Then.. I cant quite remember how we used Bugzilla 15/20 years ago :D

In T107254#9710082, @hashar wrote:

I cant quite remember how we used Bugzilla 15/20 years ago

In T107254#9710628, @Krinkle wrote:

@valerio.bozzolan The SQL statements and comments appear in contradiction. For every comment that says "Skipped bug X - state was REOPENED" there is in fact a real update statement that does exactly what the comment says it doesn't. Is this intentional?

@valerio.bozzolan The SQL statements and comments appear in contradiction. For every comment that says "Skipped bug X - state was REOPENED" there is in fact a real update statement that does exactly what the comment says it doesn't. Is this intentional?

Here our generated SQL patch candidate ready for review:

Note: we have dump of Bugzilla data at https://dumps.wikimedia.org/other/bugzilla/ , as static html file and a database dump (without emails)

And... the script now takes seconds instead of weeks. Now we have enough extra Watts to shutdown a small nuclear plant somewhere 🌈

In T107254#9706654, @Aklapper wrote:

And if it would work on a static HTML dump like ours, instead of a real BZ instance.

In T107254#9706866, @valerio.bozzolan wrote:

Have anybody already checked the file mentioned in the description, that is, maybe an useful starting point?
https://gitlab.wikimedia.org/valeriobozzolan/yet-another-bugzilla-parser/-/blob/master/data/bugzilla.csv

Anyway a required step is to take P49618 and make a nice phabricator.csv.

Have anybody already checked the file mentioned in the description, that is, maybe an useful starting point?

@valerio.bozzolan I admit I didn't try to fully understand those 400 lines which seem to cover way way more stuff than needed here. :D Like why there is an array with statuses when we only care about last Status=Resolved etc. Or why care about ACTION_WITH_LINKS etc. (Also, stuff like WORKSFORME and REMIND and LATER and --- are no statuses but resolutions.) And if it would work on a static HTML dump like ours, instead of a real BZ instance.

Small note

Had another quick look and managed to get down to have only <table>...</table> left in the local bug activity page. But then interesting problem is dealing with the rowspan: No date cell in the Status change row when several actions were performed at once and 'Status change' is not listed as first row)

In T107254#9702401, @Dzahn wrote:

git clone https://gitlab.wikimedia.org/repos/sre/miscweb/bugzilla.git

git clone https://gitlab.wikimedia.org/repos/sre/miscweb/bugzilla.git

Note: we have dump of Bugzilla data at https://dumps.wikimedia.org/other/bugzilla/ , as static html file and a database dump (without emails)

To summarize: Would need to perform 57008 times the following steps:

• take the $PhabID
• subtract 2000 from the $PhabID to get the $BzID
• pull https://static-bugzilla.wikimedia.org/show_activity.cgi?id=$BzID (e.g. $data = file_get_contents('https://fooo');)
• scrape the last (!!) line where What==Statusand Added==RESOLVED to get the When value from that line - maybe https://nyamsprod.com/blog/extracting-data-from-html-table-in-php/ or such could be handy, because need to also handle HTML table rowspan etc
• convert the When value to epoch to get $epoch (e.g. echo strtotime("2014-11-21 00:15:49 UTC");)
• UPDATE phabricator_maniphest.maniphest_task SET closedEpoch = $epoch WHERE closedEpoch IS NULL AND id = $PhabID

I obviously do not care who set the closed status (we cannot match non-existing accounts!), and I obviously do not care about creating fake transactions in the DB, but would only set that one closedEpoch column value, if at all.

Got it. In this case I'll just untag Collab.

This is not resolved; it would require altering our DB

Closing based on the last comment.

I sincerely do not remember what I was doing here :D will retry to do something during the next WMHack. I will continue from this point:

Managed access (PEBKAC).

Played with Insomnia, still having issues, filed Bitergia support request at https://support.bitergia.com/support/tickets/1183

Verified. I can login to https://wikimedia.biterg.io/identities/ ssh to wmf.sortinghat.bitergia.net

Up to @thcipriani to verify

Quoting upstream https://github.com/chaoss/grimoirelab-perceval/issues/561 :

Perceval parses everything that goes in the JSON obtained from gerrit. This is not a bug.

Also new here, kind of. :) Added you to both tickets.

In T335700#9231781, @Aklapper wrote:

• Updated ssh access request in https://support.bitergia.com/support/tickets/891 now that T335577 got resolved and there is a new (simpler) workflow
• Waiting for news about Hatstall account in https://support.bitergia.com/support/tickets/890

• Updated ssh access request in https://support.bitergia.com/support/tickets/891 now that T335577 got resolved and there is a new (simpler) workflow
• Waiting for news about Hatstall account in https://support.bitergia.com/support/tickets/890

This took a while to sort out but took an interesting turn: sortinghat (and its architecture changes) is not relevant for the usecase to perform steps locally anymore. We agreed on allowing me to create an SQL dump (instead of the previous json file to be imported locally by sortinghat into MariaDB previously) after Bitergia set up a new machine.

This is still valid. I guess the issue is finding out the right fields to use and crafting the visualization on the Biterg.io instance at https://wikimedia.biterg.io/app/kibana#/dashboard/8c515590-e1de-11e8-8aac-ef7fd4d8cbad :)

No reply, thus unfortunately closing

Let's join this lovely pastebin with our scraped stuff

Yeah having closerPHID as NULL has sense since you already filtered by maniphest_task.status != "open" AND maniphest_task.status != "stalled" AND maniphest_task.status != "duplicate" AND maniphest_task.status != "progress" and so there is no closer.

Whoa! Can I somehow surf that lovely data? 🤩 maybe in a pastebin?

Phabricator task ID = Bugzilla ticket ID + 2000

In T107254#9026648, @valerio.bozzolan wrote:

Chapter II: understanding what should be updated, from what

In T107254#9026619, @valerio.bozzolan wrote:

The script skipped all of these. It takes just the very last (→ the most meaningful) status.