@Manuel It is not that I am very much involved, but the professional situation with me is simply as it is: I can barely find any time besides the responsibilities that I carry. However, I will really make an effort in the end of May to clear up some space for our work in June. Objective constraints - that is all that I can offer as an excuse.

@BTullis With all the good will I have to help with this, there's no chance I'll be able to help before June 2023.

@MarcoSwart Unfortunately, not yet. I am struggling to find some time and promote my test server to production to serve the dashboards from there at least temporarily, until the CloudVPS issues are resolved. I will let you know as soon as I have something.

@Otourly Note: I do not work for WMDE since March 2022, so you will need to see with @Manuel about their plans, but from my side

What I will probably be able to do in the following weeks is to prepare a production environment on one my (DataKolektiv) server and run Wikidata Analytics and Wiktionary Cognate in a production environment from there, so that the community can have the dashboards available.

Ok, we have a problem.

I have found some time to inspect the problem - and the problem is very strange.

@Otourly I am currently focused on my professional obligations. I will have some more time following December 26, but then I will also need to rest and spend time with my family. So, as soon as the time permits, I will take a look at this. Also, if the issue here is related to ShinyProxy indeed - as @ItamarWMDE hypothesized - then my help is not even necessary. I do not know if anyone tried to test that hypothesis or not in the previous months, but it would definitely be my first choice in addressing why the dashboard stopped.

@LucasWerkmeister Thank you Lucas!

And the update timestamp is now Last update: 2022-11-28 09:20:32 UTC - so, WDCM is updating on the schedule.

I have checked the WDCM dashboards today (November 24, 2022) and the latest update timestamp reported was

@MisterSynergy I need to check that as soon as possible and see why the update failed (if the update failed - or it's something else that has gone wrong). Thank you for noticing this and reporting.

@ItamarWMDE @Manuel I will see to take a look at it ASAP.

@Otourly I am travelling and will be able to take a look at this after July 11.

@karapayneWMDE [Also in a recent email to @Manuel]

I notice that Goran has access to analytics_privatedata_users, is that still required?

@SLyngshede-WMF @KFrancis It's goran.s.milovanovic@gmail.com.

I have just checked the dashboard and it shows the data for all projects as it did before.

Whatever cause the problem in this dashboard's update engine described in T251792#7881692 has magically fixed itself.

This has probably something to do with the ETL module of the dashboard.

@Tobi_WMDE_SW @Manuel Here is a new problem that causes this dashboard not to update since April 20, from the dashboards' ETL/processing engine running on the stat1008 Analytics Client (crontab, every day at 06:00):

The Wiktionary Cognate dashboard is back and running from the following URL: https://wiktionary-analytics.wmcloud.org/

We are working on it, don't worry. The whole Wiktionary Cognate Dashboard is undergoing a thorough change in the way it is being deployed. Next week, I predict, the system will be back online.

@ItamarWMDE You know what I like the most? Problems that solve themselves spontaneously in time without human intervention. Namely, now I can ssh into the new CloudVPS instance, so tomorrow we can focus on the deployment of the Wiktionary Cognate Dashboard.

Also in relation to T301380:

@aborrero Thank you.

Thank you @aborrero.

@ItamarWMDE the screenshot is already shared ☝️

@rook We would really need to find out where is the problem with spinning up virtual instances in this project, and rather quickly (sorry; I am leaving my position as a Data Scientist for Wikidata and the WMDE team needs to take over my responsibilities, and we are already late in that respect, so timing is crucial here).

One hour, eight minutes right now @rook

In the old CloudVPS project wmde-dashboards I was using an g2.cores8.ram16.disk160 instance.

@ItamarWMDE Thank you. I read some suggestions on how to replace the existing fingertip, but I am really clumsy when it comes to things like this. Since we have a 1:1 scheduled for tomorrow 17:30 CET, I suggest you help me replace the fingertip in my existing SSH settings, and then we proceed to install the Wiktionary Cognate Dashboard in a new Cloud VPS project, all steps: spinning up a new virtual instance, setting up its proxy, copying the code, building the images, and deploying.

Thank you @Manuel In the meantime I have followed @Tobi_WMDE_SW advice to clean up known hosts, and did

Upon trying to connect to the wiktionary-cognate.wmde-dashboards Cloud VPS instance to fix this:

@Otourly On it.

Could you please add the following LDAP users to wmdeanalytics:

Hey @aborrero @mdipietro thank you!

The current URL is

Wiktionary Cognate Dashboard is fully re-productionized now, test server: http://datakolektiv.org/app_direct/WiktionaryCognate/

Wiktionary Cognate Dashboard is being tested under the new production approach now on the test server.

@Ragesoss You are right. I confused the format of your data with what I provided to you in the past: your datasets have user names and no user ids, and I have used user names to join them to my campaign datasets. @Hanna_Klein_WMDE needs data from one training module only: lernewikipedia training course (see: T291919#7666649). You can send the data directly to Hanna.

@Ragesoss typically needs user names to obtain the training modules data. In the past, I would share the user names with him in an email.

The user edits are shared via email.

I am on it today and it should not take long.

Here is how I think about what WMDE can do in order to address the questions posed in the ticket description.

Our Wikidata Analytics PM @Manuel would like to get in touch with the WMF Analytics team and find out if the data from the Analytics Clients are being regularly backed-up: T297485#7653025

@Manuel The WMF Analytics-Engineering team.

As long as my R production scripts on the Analytics Clients are running the data that feed our analytics will be regularly updated.

• new items out filter out added;
• a CSV download button is now present at the bottom of each table;
• check out the Dashboard.

Fixed, check out the Dashboard.

@Manuel Great, thanks!

@Manuel Could you please provide a .csv file with a list of all items that need to be filtered our? Thanks.

@Manuel Here you go: Google Drive spreadsheet with research datasets.

Nope. There are several timestamps and you are looking only at the report timestamp which is indeed updated daily because the report automatically re-renders itself every day.

@Manuel The document is useful, I agree, but trust me on this: it will be more useful to our new Data Analyst to go through all the steps in that document with me 1:1.

I would have already added new members but it turns out that I am not authorised to do so in Horizon.

Here it is: Wikidata Knowledge Transfer.

Likewise!

Thank you.

I guess this ticket should have already been resolved?

The final report is revised (section 3. Editing Behaviour) and attached:

The whole user edits ETL procedure had to be refactored and re-run in order to answer to all the newly formulated demand in this ticket.

Could you give me a timeline, when we can expect the final report?

Maybe until the end of the week. I am starting a new job full-time as of tomorrow and have an agreement on a minimal number of working hours with WMDE until March 2022 and for Wikidata related tasks only - and I still have opened, unfinished Wikidata things to work on. So, fast is not an option at this point.

From the ticket description:

@Christine_Domgoergen_WMDE Ok, at some point I will, but you really need to coordinate about priorities with @Tobi_WMDE_SW as of now. It is January 3rd and I am not supposed to invest any more hours in any analytics for the WMDE New Editors team. What you say makes sense - we started this, we should finish this, but at this point it cannot be a priority in my work. Thank you for your understanding.

Hey, no - didn't @Tobi_WMDE_SW told you that I was sick since December 25th or so?