Replace GCS + Jobs API with a proper Pub/Sub queue

[supreme-gg-gg]

I would hereby propose the final design to defeat all cloud technical debt once and for all. Yes, you heard it right!

1. All async endpoints (training, export, inference which is to be made async) will change their POST endpoints to put a message in the pub sub queue with the JSON body instead of uploading it to GCS and using gRPC to the jobs
2. The cloud run job will run, and once it completes write the result to firestore or GCS (depending on artefact type and size)

I cannot find another way to optimise step 2 other than keeping the polling method. Alternatives are complex solutions like web hook which are not necessary for such a simple use case.

[supreme-gg-gg]

Removed from title but associated with this tech debt is also replacing inference service with the router + job approach so it's more scalable (tho it seems like jobs have 5 max limit then that would be horrible)...