We're curious to know whether caching can be turned on after the superset 3 upgrade? Having trouble finding the newest task about this...

Change 709713 merged by Btullis:

[operations/puppet@production] Switch presto from Puppet to PKI certificates

https://gerrit.wikimedia.org/r/709713

Change 709737 abandoned by Btullis:

[operations/puppet@production] Add presto keytabs to the cluster coordinator replica role

Reason:

No longer needed.

https://gerrit.wikimedia.org/r/709737

Change 947857 merged by Btullis:

[operations/puppet@production] Create component/libmysql-java for bullseye

https://gerrit.wikimedia.org/r/947857

Change 947857 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Create component/libmysql-java for bullseye

https://gerrit.wikimedia.org/r/947857

Change 679295 abandoned by Hnowlan:

[analytics/aqs@master] Add docker-compose environment with cassandra

Reason:

Not needed

https://gerrit.wikimedia.org/r/679295

Looks like it is. merging in as duplicate. also see T326668

Is T330360 a duplicate of this?

Very nice fix @nfraison thank you!

Change 891555 merged by Nicolas Fraison:

[operations/puppet@production] provider_scap3: update the query to execute as the deploy_user

https://gerrit.wikimedia.org/r/891555

Seems to work fine
Before trying to redeploy analytics/hdfs-tools/deploy

Info: Unable to serialize catalog to json, retrying with pson
Info: Applying configuration version '(cb9d9be2dc) Muehlenhoff - Switch puppetdb to profile::java'
Error: Execution of '/usr/bin/scap deploy-local --repo analytics/hdfs-tools/deploy -D log_json:False' returned 70: 
Error: /Stage[main]/Profile::Analytics::Hdfs_tools/Scap::Target[analytics/hdfs-tools/deploy]/Package[analytics/hdfs-tools/deploy]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/scap deploy-local --repo analytics/hdfs-tools/deploy -D log_json:False' returned 70:  (corrective)
Notice: /Stage[main]/Profile::Analytics::Refinery::Repository/Scap::Target[analytics/refinery]/Package[analytics/refinery]/ensure: created (corrective)
Notice: /Stage[main]/Profile::Analytics::Hdfs_tools/File[/usr/local/bin/hdfs-rsync]: Dependency Package[analytics/hdfs-tools/deploy] has failures: true
Warning: /Stage[main]/Profile::Analytics::Hdfs_tools/File[/usr/local/bin/hdfs-rsync]: Skipping because of failed dependencies
Notice: /Stage[main]/Profile::Airflow/Airflow::Instance[analytics_test]/Scap::Target[airflow-dags/analytics_test]/Package[airflow-dags/analytics_test]/ensure: created (corrective)
Info: Stage[main]: Unscheduling all events on Stage[main]
Notice: Applied catalog in 60.36 seconds

In T330394#8640954, @nfraison wrote:

seems we mostly end up with same patch @jbond https://gerrit.wikimedia.org/r/891555 / https://gerrit.wikimedia.org/r/891557

I would be interested in reading/seeing how you do manual test of it. We could use an-test-client1001 for this test

Change 891557 abandoned by Jbond:

[operations/puppet@production] scap - provider: update scap provider to run git with correct user

Reason:

abanndon in favour of 891555 which also has the tests

https://gerrit.wikimedia.org/r/891557

seems we mostly end up with same patch @jbond https://gerrit.wikimedia.org/r/891555 / https://gerrit.wikimedia.org/r/891557

In T330394#8640720, @nfraison wrote:

Running the /usr/bin/git -C /srv/deployment/analytics/hdfs-tools/deploy tag --points-at HEAD manually as the user owning the folder (analytics-deploy) works:
scap/sync/2020-02-28/0001

Change 891557 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] scap - provider: update scap provider to run git with correct user

https://gerrit.wikimedia.org/r/891557

Change 891555 had a related patch set uploaded (by Nicolas Fraison; author: Nicolas Fraison):

[operations/puppet@production] provider_scap3: update the query to execute as the deploy_user

https://gerrit.wikimedia.org/r/891555

In T330394#8640754, @nfraison wrote:

Or we could ensure that the first call to get state is also run as the user owning the folder?

The git security update for safe.directory is intended exactly for that use case. A deployer could inject in the git repository some hook (as the deployment user), then when Puppet runs git as root on the repo, it might run some config or hook as root resulting in a privilege escalation.

Or we could ensure that the first call to get state is also run as the user owning the folder?

In T330394#8640720, @nfraison wrote:

Running the /usr/bin/git -C /srv/deployment/analytics/hdfs-tools/deploy tag --points-at HEAD manually as the user owning the folder (analytics-deploy) works:
scap/sync/2020-02-28/0001

While running it as root failed

nfraison@stat1008:/srv/deployment/analytics/hdfs-tools/deploy$ sudo /usr/bin/git -C /srv/deployment/analytics/hdfs-tools/deploy tag --points-at HEAD
fatal: detected dubious ownership in repository at '/srv/deployment/analytics/hdfs-tools/deploy-cache/revs/0c6e3ca61c094338d821ae7c73e244f1abb5b8bc'
To add an exception for this directory, call:

	git config --global --add safe.directory /srv/deployment/analytics/hdfs-tools/deploy-cache/revs/0c6e3ca61c094338d821ae7c73e244f1abb5b8bc

From puppet log it seems that this one is run as root and the second one as the owning user.
@hashar is that expected?

Running the /usr/bin/git -C /srv/deployment/analytics/hdfs-tools/deploy tag --points-at HEAD manually as the user owning the folder (analytics-deploy) works:
scap/sync/2020-02-28/0001

Here is the result of the debug puppet logs for one of those scap target

Debug: Executing: '/usr/bin/git -C /srv/deployment/analytics/hdfs-tools/deploy tag --points-at HEAD'
Debug: scap pkg [analytics/hdfs-tools/deploy] root=/srv/deployment/analytics/hdfs-tools, user=analytics-deploy
Debug: Executing with uid=494: '/usr/bin/scap deploy-local --repo analytics/hdfs-tools/deploy -D log_json:False'
Notice: /Stage[main]/Profile::Analytics::Hdfs_tools/Scap::Target[analytics/hdfs-tools/deploy]/Package[analytics/hdfs-tools/deploy]/ensure: created (corrective)
Debug: /Package[analytics/hdfs-tools/deploy]: The container Scap::Target[analytics/hdfs-tools/deploy] will propagate my refresh event

Updating .config file in /srv/deployment/analytics/hdfs-tools/deploy-cache in order to have git_server set to deploy1002.eqiad.wmnet instead of deploy1001.eqiad.wmnet

Change 888718 merged by Btullis:

[operations/puppet@production] Try libmariadb-java with sqoop on bullseye

https://gerrit.wikimedia.org/r/888718

Change 888718 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Try libmariadb-java with sqoop on bullseye

https://gerrit.wikimedia.org/r/888718

Cool, thanks for that write up @Ottomata. I am gonna switch to Stalled, to reflect noone is currently working on it but it's still a want to do.

I would like to see config management for Kafka topics one day, ideally integrating with EventStreamConfig (or whatever we use to declare and manage streams (and their schemas).

@Ottomata, @elukey any updates on this? Should we keep it open/close it/stall it?