Skip to content

Commit c19401e

Browse files
haarchrihaarchri
committed
feat(servingstack): Provision Vultr bare metal servers into k3s inference clusters, composing multi-vendor GPU serving stacks
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
1 parent d6e1196 commit c19401e

71 files changed

Lines changed: 6030 additions & 38 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎apis/inferenceclasses/definition.yaml‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,12 @@ spec:
4444
message: spec.provisioning.nebius is required when spec.provisioning.provider is Nebius.
4545
- rule: "self.provider != 'Vultr' || has(self.vultr)"
4646
message: spec.provisioning.vultr is required when spec.provisioning.provider is Vultr.
47+
- rule: "self.provider != 'VultrBaremetal' || has(self.vultrBaremetal)"
48+
message: spec.provisioning.vultrBaremetal is required when spec.provisioning.provider is VultrBaremetal.
4749
properties:
4850
provider:
4951
type: string
50-
enum: [GKE, EKS, AKS, Nebius, Vultr]
52+
enum: [GKE, EKS, AKS, Nebius, Vultr, VultrBaremetal]
5153
gke:
5254
type: object
5355
required: [machineType, accelerator]
@@ -242,6 +244,41 @@ spec:
242244
type: integer
243245
minimum: 1
244246
maximum: 16
247+
vultrBaremetal:
248+
type: object
249+
required: [plan, accelerator]
250+
properties:
251+
plan:
252+
type: string
253+
description: >-
254+
Vultr bare metal plan ID (e.g.
255+
vbm-256c-3072gb-8-mi355x-gpu). The plan
256+
determines the GPU model and count; the
257+
accelerator block below is informational.
258+
minLength: 1
259+
maxLength: 63
260+
accelerator:
261+
type: object
262+
description: >-
263+
GPU accelerator the plan carries. Provisioning
264+
input only: the scheduler matches against
265+
spec.devices, not this block. The type's vendor
266+
prefix (amd-, nvidia-) selects the GPU node
267+
taint and labels.
268+
required: [type, count]
269+
properties:
270+
type:
271+
type: string
272+
description: >-
273+
GPU accelerator type (e.g. amd-mi355x,
274+
nvidia-h100). Reported on the consuming
275+
InferenceCluster's status.
276+
minLength: 1
277+
maxLength: 63
278+
count:
279+
type: integer
280+
minimum: 1
281+
maximum: 16
245282
devices:
246283
type: array
247284
description: >-

‎apis/inferenceclusters/definition.yaml‎

Lines changed: 129 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ spec:
3232
x-kubernetes-validations:
3333
- rule: "self.cluster.source != 'Nebius' || !has(self.nodePools) || self.nodePools.all(p, !has(p.fabric) || p.fabric.type != 'InfiniBand' || has(p.fabric.infiniband))"
3434
message: fabric.infiniband is required when fabric.type is InfiniBand and cluster.source is Nebius.
35+
- rule: "self.cluster.source != 'VultrBaremetal' || !has(self.nodePools) || self.nodePools.all(p, !has(p.maxNodeCount))"
36+
message: node pools cannot autoscale when cluster.source is VultrBaremetal; bare metal pools are fixed size.
37+
- rule: "self.cluster.source != 'VultrBaremetal' || !has(self.stack) || self.stack == 'Standard'"
38+
message: only the Standard stack is supported when cluster.source is VultrBaremetal.
3539
properties:
3640
cluster:
3741
type: object
@@ -49,12 +53,14 @@ spec:
4953
message: spec.cluster.nebius is required when spec.cluster.source is Nebius.
5054
- rule: "self.source != 'Vultr' || has(self.vultr)"
5155
message: spec.cluster.vultr is required when spec.cluster.source is Vultr.
56+
- rule: "self.source != 'VultrBaremetal' || has(self.vultrBaremetal)"
57+
message: spec.cluster.vultrBaremetal is required when spec.cluster.source is VultrBaremetal.
5258
properties:
5359
source:
5460
type: string
5561
description: >-
5662
Cluster provisioning method.
57-
enum: [GKE, EKS, AKS, Nebius, Vultr, Existing]
63+
enum: [GKE, EKS, AKS, Nebius, Vultr, VultrBaremetal, Existing]
5864
existing:
5965
type: object
6066
description: >-
@@ -310,6 +316,128 @@ spec:
310316
default: default
311317
minLength: 1
312318
maxLength: 253
319+
vultrBaremetal:
320+
type: object
321+
description: >-
322+
Vultr bare metal (k3s) cluster configuration.
323+
Required when source is VultrBaremetal. Provisions
324+
bare metal servers - one CPU-only management server
325+
plus the GPU pools - and installs a k3s cluster
326+
onto them over SSH. Bare metal has no autoscaling,
327+
so pools are fixed size, and provisioning takes
328+
tens of minutes.
329+
required: [region, ssh]
330+
properties:
331+
region:
332+
type: string
333+
description: >-
334+
Vultr region for all servers (e.g. ewr, ord).
335+
Bare metal plan availability varies by region;
336+
check with vultr-cli plans list --type vbm.
337+
minLength: 1
338+
maxLength: 32
339+
management:
340+
type: object
341+
default: {}
342+
description: >-
343+
The CPU-only bare metal server that runs the
344+
k3s server (the management plane).
345+
properties:
346+
plan:
347+
type: string
348+
default: vbm-6c-32gb-amd
349+
description: >-
350+
Vultr bare metal plan for the management
351+
server. The default is a CPU-only plan
352+
available in most regions that offer bare
353+
metal.
354+
minLength: 1
355+
maxLength: 63
356+
osId:
357+
type: integer
358+
default: 2284
359+
description: >-
360+
Vultr operating system ID installed on
361+
every server. Defaults to Ubuntu 24.04 LTS
362+
x64; list IDs with vultr-cli os list.
363+
ssh:
364+
type: object
365+
description: >-
366+
SSH key pair used to reach the servers. The
367+
public key is registered with Vultr and
368+
installed on every server; the private key
369+
drives the k3s install over SSH.
370+
required: [secretRef]
371+
properties:
372+
secretRef:
373+
type: object
374+
description: >-
375+
Secret holding the SSH key pair. The Secret
376+
must exist in the modelplane-system
377+
namespace.
378+
required: [name]
379+
properties:
380+
name:
381+
type: string
382+
minLength: 1
383+
maxLength: 253
384+
privateKeyKey:
385+
type: string
386+
default: ssh-privatekey
387+
description: >-
388+
Key within the Secret that holds the
389+
private key.
390+
minLength: 1
391+
maxLength: 253
392+
publicKeyKey:
393+
type: string
394+
default: ssh-publickey
395+
description: >-
396+
Key within the Secret that holds the
397+
public key.
398+
minLength: 1
399+
maxLength: 253
400+
username:
401+
type: string
402+
default: root
403+
description: >-
404+
SSH user the servers accept the key for.
405+
Vultr installs keys for root by default.
406+
minLength: 1
407+
maxLength: 63
408+
k3s:
409+
type: object
410+
default: {}
411+
description: The k3s release installed on the servers.
412+
properties:
413+
channel:
414+
type: string
415+
default: v1.34
416+
description: >-
417+
k3s release channel. Defaults to the first
418+
channel where Dynamic Resource Allocation
419+
(how GPUs bind to pods) is generally
420+
available.
421+
minLength: 1
422+
maxLength: 32
423+
credentials:
424+
type: object
425+
description: >-
426+
Vultr ProviderConfig or ClusterProviderConfig used to
427+
authenticate to the Vultr API. Defaults to the
428+
ClusterProviderConfig named default.
429+
properties:
430+
type:
431+
type: string
432+
default: ClusterProviderConfig
433+
enum:
434+
- ProviderConfig
435+
- ClusterProviderConfig
436+
name:
437+
type: string
438+
default: default
439+
minLength: 1
440+
maxLength: 253
313441
stack:
314442
type: string
315443
default: Standard

‎apis/k3sclusters/composition.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: apiextensions.crossplane.io/v1
2+
kind: Composition
3+
metadata:
4+
name: k3sclusters.infrastructure.modelplane.ai
5+
spec:
6+
compositeTypeRef:
7+
apiVersion: infrastructure.modelplane.ai/v1alpha1
8+
kind: K3sCluster
9+
mode: Pipeline
10+
pipeline:
11+
- functionRef:
12+
name: modelplane-modelplanecompose-k3s-cluster
13+
step: compose-k3s-cluster

0 commit comments

Comments
 (0)