Support kubernetes compatible configs in the jinja templates

[ArnavBalyan]

Issue:

• Currently we convert Skyshift entities to K8s using pre-defined j2 templates.
• These templates hold a limited set of fields which can be substituted by Skyshift. To support anything that is not already included in the templates, we need to either:
  • Create a kubernetes config file and directly apply it (bypassing Skyshift)
  • Or, update our template to support it from Skyshift side.
• We will have to keep these templates upto date and make them extensive to support almost everything that kubernetes supports which may not always be possible.

Possible Workaround:

• To reduce the friction, would it be a good idea to allow users to pass K8s supported properties under a new section something like native_configs?
• Any anything passed under this field, may be directly appended to the final converted yaml sent to kubernetes.
• Values under this field would be strictly kubernetes specific and may not require any validation from skyshift end. (allowing users to passs custom requirements and anything that skyshift does not support)

cc @michaelzhiluo was not sure if this is a valid problem or needs an immediate solution, just wanted to know if there could be something done. Thanks!

[michaelzhiluo]

The templates for jobs are intended to be compatibility layer components across cluster managers such as K8, Slurm, and Ray. There will always be different/unique aspects for each cluster manager; for example, K8 can support multiple containers per pod.

I completely agree with your proposition and this was also proposed by @dmatch01.

The compatibility layer should ultimately involve a translation and detranslation components. If a K8 yal is submitted, it is first converted into a Skyshift yaml, and then translated back into whatever cluster manager it is submitted to. However, I think this can be done after the open-source milestone, and then it can be part of the milestones.

[ArnavBalyan]

Thanks, this makes a lot of sense. Yes maybe the native_configs could be resource manager specific. Yes for sure can be done after open sourcing thanks for taking a look!