Skip to content

Commit 8fff2e0

Browse files
committed
docs: define V6 elastic deployment model
1 parent b5ed54d commit 8fff2e0

1 file changed

Lines changed: 117 additions & 0 deletions

File tree

‎docs/architecture/V6_HYPERSCALE_ARCHITECTURE_PLAN.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,123 @@ purchase specification. The all-one-minute workload is expected to require
417417
approximately 700-1,500 application vCPUs depending on strategy CPU cost and
418418
completion SLO.
419419

420+
## Deployment packaging and elastic expansion
421+
422+
V6 must be deployable at small scale without creating a second architecture
423+
that has to be replaced during growth. The same versioned service images,
424+
configuration schema, event contracts, database migrations, health checks, and
425+
operational controls will be used in every supported deployment profile.
426+
427+
| Profile | Intended use | Runtime shape |
428+
| --- | --- | --- |
429+
| Developer | Local development and deterministic integration tests | Docker Compose with single-replica services and replaceable local dependencies |
430+
| Production starter | Small self-hosted or managed installation | Container platform with at least two API replicas, dedicated worker roles, managed PostgreSQL and Redis-compatible state |
431+
| Hyperscale | Multi-AZ managed production | Kubernetes, managed event backbone, partitioned worker pools, multi-AZ data services, and separate live and batch capacity |
432+
433+
Docker Compose remains a development and small-installation package. It is not
434+
the 100,000-strategy scheduler. Production starter and hyperscale deployments
435+
must run the same application artifacts with different replica counts and
436+
infrastructure adapters rather than environment-specific application forks.
437+
438+
### Release artifacts and infrastructure as code
439+
440+
Each V6 release will publish immutable, content-addressed images for explicit
441+
process roles. A single source revision may produce role-specific images or a
442+
shared image with fixed entrypoints, but runtime command selection must be
443+
declarative and reviewable. Mutable `latest` tags are excluded from production
444+
rollouts.
445+
446+
The deployment repository will provide:
447+
448+
- Terraform or OpenTofu modules for network, Kubernetes, database, event
449+
backbone, hot-state service, object storage, workload identity, and secrets;
450+
- a versioned Helm chart, or an equivalent declarative package, with separate
451+
deployments for API, push gateway, market ingestion, evaluators, level
452+
engines, execution gateways, reconciliation, notifications, and batch jobs;
453+
- checked-in environment overlays for development, staging, and production,
454+
containing capacity and routing differences but no secrets;
455+
- migration jobs, readiness checks, dashboards, alerts, disruption budgets,
456+
topology-spread constraints, and network policies as release artifacts;
457+
- a rendered-manifest validation step in CI so configuration failures are found
458+
before a cluster rollout.
459+
460+
Secrets are referenced through workload identity and a managed secret store.
461+
They are not embedded in images, Helm values, event payloads, or GitOps state.
462+
463+
### Stateless and stateful boundaries
464+
465+
API, push, ingestion, evaluator, level-engine, and execution processes must be
466+
replaceable without relying on a pod filesystem or process-local ownership.
467+
Durable state belongs to PostgreSQL, the event log, object storage, or a
468+
versioned checkpoint store. Hot state may be cached in a partition owner, but
469+
the owner must be fenced and recoverable from a checkpoint plus replay.
470+
471+
This boundary allows a deployment to add servers by increasing replicas and
472+
consumer capacity instead of copying databases or manually assigning users to
473+
machines. Kubernetes service discovery balances stateless service calls, while
474+
event partitions and leases assign stateful work.
475+
476+
### Horizontal expansion procedure
477+
478+
Normal expansion will follow this order:
479+
480+
1. provision or enlarge the appropriate node pool through infrastructure code;
481+
2. add service replicas while keeping them unready for workload admission;
482+
3. restore caches, establish venue streams, and pass dependency health checks;
483+
4. join the relevant consumer group and transfer partitions through the drain
484+
and fencing protocol;
485+
5. verify queue age, evaluation latency, reconciliation health, and error budget;
486+
6. admit additional strategies only after measured spare capacity is available.
487+
488+
Scale-in reverses this sequence. A worker first stops accepting new ownership,
489+
checkpoints dirty state, commits safe offsets, transfers partitions, and only
490+
then terminates. Forceful process termination is tested as a recovery case but
491+
is not the normal scale-in mechanism.
492+
493+
Increasing replica count does not increase concurrency past the available event
494+
partitions. Partition expansion is therefore a planned capacity operation. It
495+
must preserve key ordering, use a compatible partitioning scheme, and include a
496+
rebalancing plan. Account execution partitions require particular care because
497+
changing ownership cannot permit two workers to submit orders concurrently for
498+
the same credential.
499+
500+
### Rollout, compatibility, and rollback
501+
502+
The platform will use rolling or canary deployment with a small partition set
503+
or tenant cohort before broad rollout. A release may consume the current and
504+
immediately preceding event and checkpoint schema during the migration window.
505+
Producers switch only after compatible consumers are healthy.
506+
507+
Database migrations use expand-and-contract changes. Destructive schema removal
508+
occurs only after old application versions are drained and rollback is no
509+
longer required. A rollback must restore the previous application version
510+
without rolling back acknowledged orders, fills, ledger entries, offsets, or
511+
fencing epochs.
512+
513+
Live execution has stricter rollout policy than stateless APIs and batch pools:
514+
515+
- execution gateways retain minimum replicas and zone diversity throughout a
516+
deployment;
517+
- one credential partition has exactly one fenced owner at a time;
518+
- canary execution begins with signal-only or designated test accounts;
519+
- automated rollback is disabled for ambiguous order-submission failures until
520+
reconciliation determines the venue result;
521+
- emergency stop remains reachable independently of the release being rolled
522+
out.
523+
524+
### Portable operating contract
525+
526+
The first production reference may use one cloud provider, but application
527+
services will depend on portable contracts: OCI images, Kubernetes APIs,
528+
PostgreSQL, Kafka-compatible ordered streams, S3-compatible object storage, and
529+
a Redis-compatible hot-state interface. Provider-specific managed services are
530+
selected behind these contracts.
531+
532+
Portability does not require identical infrastructure on every provider. It
533+
requires that a new region or provider can be created from versioned modules,
534+
receive a tested configuration, restore durable state, and pass the same
535+
conformance and load tests without application code changes.
536+
420537
## Load balancing and autoscaling
421538

422539
HTTP load balancing and strategy workload distribution are different problems.

0 commit comments

Comments
 (0)