[Thread] Open-Sourcing SkyShift

[michaelzhiluo]

These are the following steps to open-source SkyShift:

• Docs

  • Docs for Services @pr2satish @colinc9 @ArnavBalyan (Docs for SkyShift Services #312, In PR)
  • Polish/Readjust the Docs; Typos, Potential Reorganization of Different Pages (Everyone)
  • Docs for the Cloud Feature @just-in-chang (Cloud Docs #307)
  • Fixing Slurm Documentation @dmatch01 @DC-Base (Slurm docs #317, In PR)
  • Ray Documentation @AlexCuadron (In PR, [Ray] Added quickstart documentation #313)
  • Docs on Scheduler and Scheduling Plugins @dmatch01 (Added initial outline for scheduler documentation. #299)
  • Automatic Documentation for our Python API and CLI. @ArnavBalyan (Setup Auto Documentation for SkyShift CLI #298)
  • Release Notes @michaelzhiluo

• Features

  • Ray Test PR @AlexCuadron (In PR, [Ray] Tests for ray #230)

• Clean up README.md @michaelzhiluo (Done!)

• Github actions for website.

• Create roadmap.md @MEllis-github @dmatch01 (Add roadmap #296 , Update ROADMAP.md #315 )

• Close issues or mark as stale

• Create SkyShift examples for AI serving, AI training, and different workloads. @AlexCuadron @praveingk @michaelzhiluo @colinc9 (Every PR will need documentation under examples/)

  • Multiple vLLMs across clusters @praveingk @michaelzhiluo (Add vllm deployment example #319, In PR)
  • Simple Spark Application @ArnavBalyan (Add example for Data Processing/ETL with Spark #300)
  • SparkSQL with TPCDS/TPCH @ArnavBalyan ([SkyShift] Add docs for multi executor Spark job - TPCDS #316, In PR)
  • Microservice across clusters (BookInfo) @praveingk (Bookinfo microservice multi cluster demo #314)
  • RBAC Example (how to add users and create new roles for them) @ArnavBalyan (Add Documentation and Examples for RBAC #305)

[praveingk]

We need a logo for Skyshift right?

[MEllis-github]

• contributor guide

Also, a goal or milestone w.r.t. the testing/robustness discussion might be beneficial.