[foreman*|pulp|candlepin] support foremanctl deployments

[pmoravec]

A merge-able implementation of #4189. When reviewing all stuff being collected in foreman plugin, I did remove some parts that are not further relevant (passenger, qpid).

I am kindy asking @evgeni / @adamruzicka / @ehelms for a review of the TODO questions as I dont know how (or if) to collect some stuff.

All other plugin changes (pulp, candlepin,..) included, as well as a new foremanctl plugin is implemented.

---

Please place an 'X' inside each '[]' to confirm you adhere to our Contributor Guidelines

• Is the commit message split over multiple lines and hard-wrapped at 72 characters?
• Is the subject and message clear and concise?
• Does the subject start with [plugin_name] if submitting a plugin patch or a [section_name] if part of the core sosreport code?
• Does the commit contain a Signed-off-by: First Lastname email@example.com?
• Are any related Issues or existing PRs properly referenced via a Closes (Issue) or Resolved (PR) line?
• Are all passwords or private data gathered by this PR obfuscated?

[packit-as-a-service]

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

• sudo dnf install -y 'dnf*-command(copr)'
• dnf copr enable packit/sosreport-sos-4330
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[adamruzicka]

Ideally also all of these

# systemctl list-units | grep dynflow-sidekiq@ | awk '{ print $1 }'
dynflow-sidekiq@orchestrator.service
dynflow-sidekiq@worker-hosts-queue.service
dynflow-sidekiq@worker.service

Or maybe even better, logs of all the services forming foreman.target?

# systemctl list-dependencies foreman.target | grep -Po '[a-zA-Z].*'
foreman.target
candlepin.service
dynflow-sidekiq@orchestrator.service
dynflow-sidekiq@worker-hosts-queue.service
dynflow-sidekiq@worker.service
foreman-proxy.service
foreman-recurring@daily.service
foreman-recurring@daily.timer
foreman-recurring@hourly.service
foreman-recurring@hourly.timer
foreman-recurring@monthly.service
foreman-recurring@monthly.timer
foreman-recurring@weekly.service
foreman-recurring@weekly.timer
foreman.service
postgresql.service
pulp-api.service
pulp-content.service
pulp-worker@1.service
pulp-worker@2.service
pulp-worker@3.service
pulp-worker@4.service
pulp-worker@5.service
pulp-worker@6.service
redis.service
pulp-worker.target
pulp-worker@1.service
pulp-worker@2.service
pulp-worker@3.service
pulp-worker@4.service
pulp-worker@5.service
pulp-worker@6.service

[pmoravec]

We collect self.add_journal(units="dynflow-sidekiq@*") on line 179. candlepin, foreman-* and pulp-* shall be rather collected by the respective plugins (so I will extend the foreman_proxy one, candlepin and pulp is on the TODO list).

both redis and postgresql journals make sense to collect only in foremanctl, right? It makes bit more sense to collect them in foreman plugin as a special use case, since redis and postgresql plugins dont collect the journals by default.

[adamruzicka]

Would it be possible to pipe the two together in one go? I'm pretty sure it could be done in a hack way, this question is more about doing it cleanly.

[pmoravec]

@TurboTurtle some thoughts? I know there are some know issues when collecting commands with pipes, so we try to avoid that as much as we can.

Moreover, plain:

self.add_cmd_output(f'podman exec postgresql {_cmd} | podman exec -i foreman /usr/libexec/dynflow-expand', env=self.env,suggest_filename=f'piped_{dyn}', timeout=600)

does not work (psql: invalid option -- 'i'), regardless if I add - to tail of the 2nd podman command, and:

podman exec -i foreman /usr/libexec/dynflow-expand < <(podman exec postgresql {_cmd})

gets stuck.

I dont like the proposed approach, but dont see anything better (that would work).

[pmoravec]

Kindly asking @arif-ali and/or @TurboTurtle for a review - I would like to fix this regression till the next release scheduled for the next week.