Enclave-Enforced Migration Cooldown

[aruokhai]

Problem

The migration cooldown—the operator's reaction window before the old enclave releases its secrets to a new (possibly attacker-controlled) PCR0—is currently enforced only by the host supervisor.

Specifically:

• supervisor/migrate.go implements runCooldown, which waits using the host clock and the host-writable SSM parameter MigrationRequestedAt.
• The enclave's phase-2 migration handler (runtime/servers.go) performs no cooldown verification. StartMigration (runtime/migrate.go) immediately:
  1. Commits PCR31.
  2. Creates the migration KMS key bound to {oldPCR0, newPCR0}.
  3. Re-encrypts every secret.
  4. Switches KMSKeyID.

A compromised host can therefore bypass the cooldown entirely by invoking the phase-2 endpoint directly. The existing cooldown is operational guidance rather than a security control.

---

Goal

Move cooldown enforcement into the old enclave itself.

The enclave must refuse to release secrets until the configured cooldown has elapsed, using a notion of time that the host cannot manipulate.

The design must guarantee that a compromised host may:

• delay migration,
• abort migration,
• deny service,

but cannot shorten the cooldown, regardless of:

• SSM manipulation,
• enclave restarts,
• replayed artifacts,
• direct calls to the phase-2 (finalise-migration) endpoint.

Every migration request must also be recorded in a host-independent, tamper-proof trail that a thirdparty can inspect and verify.

---

Trusted Time

The enclave requires a notion of time that the host cannot manipulate. The design uses two trusted time sources:

• S3 LastModified of the intent object (earliest valid version) — assigned server-side by S3 when the request is written to the bucket — as the authoritative timestamp for when the request became observable.
• The enclave's trusted PTP-backed clock (time.Now(), dependent on Fix feat(runtime): correct enclave clock drift via hypervisor PTP clock #138) as the authoritative value for current time.

S3 sets LastModified at write time and exposes no way to backdate it, so the host cannot make a request look older than the moment it actually appeared in the bucket. This is deliberately not the attestation's own timestamp: that records when the document was generated, which the host can trigger privately — running the enclave to mint an attestation and holding it off-trail — so it does not bound the reaction window. The window must count from publication (the observable S3 write), which is what LastModified measures. Taking the earliest valid version defeats replay: re-uploading a captured intent only ever creates a newer version, never an older timestamp.

The cooldown duration (ENCLAVE_MIGRATION_COOLDOWN) is baked into the EIF and included in nonOverridableEnv (runtime/environment.go), making it PCR0-measured. Modifying the duration changes PCR0 and therefore invalidates access to the migration KMS key.

---

Threat Model

The host shares the enclave's IAM role and controls the vsock transport. Consequently it can:

• upload new objects,
• replay previously captured attestation documents,
• restart the enclave,
• repoint the MigrationAnchorBucketName SSM parameter,
• invoke the migration vsock endpoints directly (it is the only party that can reach them),
• block access to S3.

It cannot:

• forge a valid intent attestation bound to the legitimate enclave's PCR0 — the host may boot its own malicious EIF, but that enclave has a different PCR0 and its attestation is rejected,
• tamper with an intent's committed fields without breaking the AWS-Nitro signature,
• backdate an intent's S3 LastModified — it is assigned server-side at write time, so the host cannot make a request appear older than when it actually landed in the bucket,
• delete compliance-locked objects,
• repoint the bucket undetected (the bucket identity is committed in state_root; see Bucket Binding).

The only replay available is re-uploading a previously captured intent attestation. user_data binds each document to a specific sequence number, so a replay cannot be relabelled to a higher (winning) sequence; and re-uploading only mints a newer object version, which cannot lower the request time because the protocol reads the earliest valid version's LastModified.

The protocol therefore derives the request time from S3's server-assigned LastModified — the moment the request became observable — not from the attestation's own generation timestamp, which the host could set privately and hold off-trail.

---

Design

Migration becomes a two-phase protocol enforced entirely by the old enclave, and its control plane moves off the public listener onto a dedicated parent-only vsock channel (see Migration Control Channel).

Intent state is stored as NSM-attested objects in a new compliance-locked migration bucket. Each object is a COSE Sign1 attestation document signed by the AWS Nitro root, produced with nsm.BuildAttestationDocument(WithUserData(...)) — the same machinery already used for state-origin receipts. Authenticity and integrity come from the attestation signature

The design reuses from runtime/anchor.go:

• the S3 object-versioning / ListObjectVersions scan pattern,
• canonical CBOR encoding for the user_data payload.

Migration Intent

Intent objects use the key:

migration-intent/<new_pcr0>/<seq %020d>

Each object is an NSM attestation document whose user_data is the canonical-CBOR encoding of:

migrationIntentV1{
    Schema, \\  For example `enclave.migration_intent.v1`,
    NewPCR0,
    Seq,
    Action, // requested | aborted
}

The user_data binds the intent identity — <deployment>/<app>/migration-intent/<new_pcr0>/<seq> — so a genuine attestation cannot be re-presented under a different object key.

Object Lock retention matches the freshness anchor: the 10-year ENCLAVE_ANCHOR_WINDOW, compliance mode — immutable even to account root. The long window exists so the bucket is a durable, tamper-proof trail of every migration request and abort

---

Winning Record

For each destination PCR0:

• Ignore versions that fail attestation verification (bad signature, wrong Nitro root, or a PCR0 that does not match the finalising enclave's own PCR0) or whose user_data disagrees with the object key.
• Select the highest sequence number containing at least one valid version.
• Read RequestedAt as the earliest LastModified among the valid versions of that sequence.

This prevents replay attacks from resetting the cooldown.

An aborted tombstone becomes the newest record and blocks migration.

Submitting another request creates seq + 1, producing a fresh cooldown window rather than shortening the existing one.

---

Bucket Binding (state_root)

The intent store and the freshness anchor both resolve their bucket from the host-writable SSM parameter AnchorBucketName (runtime/anchor.go). Left unattested, a privileged principal could repoint it to a bucket they control — one the operator does not inspect, and which they may have pre-seeded with an aged, captured intent — defeating both the reaction window and the request trail.

The bucket identity — {bucket name, owning account ID, region} — is therefore committed into stateRootInputV1 (runtime/state_origin.go), so a repoint changes the attested state_root and fails receipt verification closed on resume.

---

Cooldown Gate

The first statement of the phase-2 finalisation (StartMigration) becomes:

enforceCooldownGate(ctx, newPCR0)

before CommitPCR(31, ...).

If:

cooldown <= 0

the gate is skipped, preserving today's development behavior.

Otherwise the enclave:

1. Reads the winning intent from S3 and verifies its attestation (signature, Nitro root, PCR0 == own PCR0).
2. Reads the current time from its trusted PTP-backed clock (time.Now()).
3. Computes:

elapsed = max(0, now - RequestedAt)

Migration proceeds only if:

action == requested &&
cooldown <= elapsed 

Errors:

• ErrMigrationNotRequested
• ErrCooldownActive
• ErrIntentAborted

---

Migration Control Channel

Migration control moves off the external HTTPS mux onto a dedicated vsock channel that only the parent instance can dial. External clients (app users, the internet) cannot reach it at all; the only possible caller is the supervisor.

The channel listens for exactly two POST requests:

• request-migration — writes an intent record for new_pcr0. Its action field selects requested (open or refresh the cooldown window) or aborted (write the tombstone that cancels a pending migration). Returns the current cooldown status.
• finalise-migration — the phase-2 command (formerly start-migration). Runs enforceCooldownGate and, only if it passes, releases secrets to new_pcr0.

---

Security Properties

A compromised host cannot shorten the cooldown because:

• request times are S3's server-assigned LastModified (earliest version), which the host cannot backdate,
• current time comes from the enclave's PTP-backed clock, independent of the host,
• cooldown duration is PCR0-measured,
• the intent bucket identity is committed in the attested state_root, so the host cannot substitute or repoint it,
• intent state survives enclave restarts,
• replayed intents cannot be relabelled to a winning sequence (user_data binds the seq),
• invalid or foreign-PCR0 objects fail attestation verification,
• abort tombstones are 10-year compliance-locked and cannot be deleted or revived,
• S3 failures fail closed.

The host retains only denial-of-service capabilities—it may delay or abort migration, but cannot cause secrets to be released early. Because the vsock channel is parent-only, non-host callers cannot invoke migration at all.

---

Decisions

• Remove the legacy SSM-based cooldown entirely.
• Move the migration control plane to a dedicated parent-only vsock channel exposing request-migration and finalise-migration. If the enclave does not expose this channel, the supervisor fails with a clear upgrade-required error.
• Remove runCooldown and the MigrationRequestedAt SSM parameter.
• Intents are attestation-backed, .
• Intents use the same 10-year compliance retention as freshness anchors, giving a durable request trail, and the shared bucket identity is committed to state_root.

---

Implementation

1. runtime/migration_intent.go

Introduce a new MigrationIntentStore, modeled after runtime/anchor.go but attestation-backed.

Responsibilities:

• request migration (write a requested intent attestation),
• abort migration (write an aborted tombstone attestation),
• retrieve newest intent,
• retrieve newest live intent,
• verify intent attestations (Nitro root + PCR0 == own) and reject key/user_data mismatches,
• enforce replay resistance via earliest-valid-version LastModified,
• set 10-year (ENCLAVE_ANCHOR_WINDOW) compliance retention on each object,

---

2. runtime/migrate.go

• Add MigrationIntentStore to Migrator.
• Implement:
  • RequestMigration() (handles both requested and aborted actions),
• Add enforceCooldownGate() as the first step of StartMigration (the phase-2 finalisation).
• Rewrite CooldownStatus() to derive state from migration intents instead of SSM.
• Remove migrationRequestedAtParam().

---

3. Migration vsock channel + runtime/servers.go

Expose on the dedicated parent-only migration vsock channel:

• POST request-migration
• POST finalise-migration

Map cooldown failures to:

HTTP 425 Too Early

Continue exposing cooldown status through /v1/enclave-info (read-only, external mux) using the existing JSON fields.

---

4. runtime/runtime.go + runtime/state_origin.go

• Construct the MigrationIntentStore after S3 initialization and inject it into NewMigrator(). Stand up the migration vsock listener during boot.
• Commit {AnchorBucketName, owning account ID, region} into stateRootInputV1 (runtime/state_origin.go) — the same binding used for the freshness anchor, so a repoint fails receipt verification closed.

---

5. supervisor/migrate.go

Remove:

• runCooldown
• MigrationRequestedAt
• supervisor cooldown environment wiring

Replace with, over the migration vsock channel:

1. POST request-migration (action: requested).
2. Poll remaining cooldown using the returned status.
3. On cancellation, best-effort POST request-migration (action: aborted).
4. POST finalise-migration only after the cooldown expires.

The enclave performs its own verification regardless of supervisor behavior.

---

6. Infrastructure

No IAM changes are required (the shared anchor-bucket grant already covers intents).

Remove:

• aws_ssm_parameter.migration_requested_at
• associated IAM permissions

Document:

• the migration-request trail: how to list the anchor bucket's migration-intent/ prefix and verify each intent out-of-band against the AWS Nitro root,
• HTTP 425 behavior,
• request → wait → finalise migration workflow.

---

7. Tests

Add coverage for:

• intent round-trip,
• replay attacks (re-uploaded attestation as a newer version),
• foreign-PCR0 intent rejection,
• idempotent requests,
• abort tombstones,
• expiration,
• attestation verification failure,
• bucket-repoint fails closed on resume (state_root binding),
• cooldown gate,
• HTTP 425 responses,
• direct finalise-migration during cooldown,
• vsock channel reachability (parent-only),
• supervisor polling behavior.

---

Implementation Order

1. Migration intent store (attestation-backed, 10-year retention).
2. Cooldown gate.
3. Runtime wiring + migration vsock channel + state_root bucket binding.
4. vsock endpoints (request-migration, finalise-migration).
5. Supervisor changes.
6. Terraform cleanup.
7. Documentation.
8. Integration tests.

---

Verification

1. Run:

go build ./...
go test ./runtime ./supervisor

2. Execute the full integration suite:

make test-build

cd test

docker compose --profile test build test-runner

docker compose --profile test run -T test-runner

Expected behavior:

• pending cooldown survives restarts,
• abort continues to work,
• successful migration after cooldown,
• rollback still functions,
• direct finalise-migration returns HTTP 425 while the cooldown is active,
• the migration endpoints are unreachable from outside the parent instance,

---

Out of Scope

• Authenticating the migration endpoints beyond channel isolation. They live on a parent-only vsock channel; the enclave-side cooldown gate—not authentication—is what makes host invocation safe.

[chris-ricketts]

Additional Context

The current proposal provides the first two properties. The third is an additional hardening:

1. Same-sequence races are idempotent. The earliest valid version wins. Later versions are ignored.
2. Entries cannot be moved to a later sequence. The sequence is covered by the Nitro signature and must match the object key. Copying an attestation to another sequence fails validation.
3. Proposed addition — an expired request is final. Enclave code refuses to attest or write a new sequence when the previous winning intent is requested and its cooldown has expired.

The trade-off is irreversible finality: if the selected successor is faulty in a way that prevents its migration machinery from working, the old enclave cannot redirect to another successor and the upgrade path may be permanently stuck.

It is assuming that a verifier can only observe the current migration log head state via /v1/enclave-info.

This addition may not be required if a public enclave endpoint exposes the entire migration log, or if the intent bucket is externally publicly readable: the verifier could then independently detect new intents originating from old PCR0s during their cooldown and exit the system.

The enclave, not the caller, assigns the next sequence and attests:

user_data = cbor{schema, bucket, seq, requested|aborted, next_pcr0}

Example:

Both enclaves observe next sequence = 7

Public A   REQUEST B  ──► /7, version v1 @ t0  ──► canonical
Clone A    ABORT      ──► /7, version v2 @ t1  ──► ignored
Host       replay v1  ──► /7, version v3 @ t2  ──► ignored
Host       copy v2    ──► /8                    ──► invalid: abort attests seq 7

sequence 7 = REQUEST B
RequestedAt = t0
deadline = t0 + cooldown

before deadline  → enclave may attest sequence 8
at/after deadline → enclave refuses all new sequence entries

The proposed addition prevents a hidden clone of an old image from opening a new migration after the public deployment has moved on:

Public chain:

    A ──request B; cooldown expires──► B ──► C ──► D  (current PCR0)
                                                       │
                                                       └─ /v1/enclave-info
                                                          reports D only

Hidden fork:

    host-retained clone A
             │
             │ tries to attest: REQUEST EVIL, sequence 1
             ▼
    A's winning intent: sequence 0, REQUEST B
    now >= RequestedAt + cooldown
             │
             ▼
    REFUSE to create the attestation or write sequence 1
             │
             └─ no A → EVIL cooldown, finalisation, or secret release

Without this addition, the clone could create A → EVIL several images behind the public head. If /v1/enclave-info reports migration status only for the current PCR0, the verifier would not see that cooldown before the clone finalised and released its copy of the secrets.

The sequence namespace and winner lookup must therefore be scoped to the source PCR0, not independently to each destination. Otherwise A → B and A → EVIL occupy separate logs, and the expired A → B request cannot seal A:

migration-intent/<source_pcr0>/<seq>

next_pcr0 remains in the attested payload.

flowchart TD
    New["Request new intent"] --> Resolve["For this source PCR0:<br/>resolve highest sequence and earliest valid version"]
    Resolve --> Previous{"Previous winner"}
    Previous -- "None or ABORTED" --> Next["Choose next sequence"]
    Previous -- "REQUESTED" --> Expired{"now >= RequestedAt + cooldown?"}
    Expired -- "Yes" --> Refuse["Proposed addition:<br/>refuse to create attestation"]
    Expired -- "No" --> Next
    Next --> Attest["Attest bucket, sequence,<br/>action, and next PCR0"]
    Attest --> Write["Write sequence key"]
    Write --> Race["Earliest valid version wins;<br/>later versions are ignored"]

Loading

Protocol pseudocode:

resolve winning intent for this source PCR0:
    discard versions with invalid attestations or key/data mismatches
    seq = highest sequence containing a valid version
    winner = earliest valid version at seq
    return winner and LastModified

write new intent(action, next_pcr0):
    previous = resolve winning intent

    // Proposed addition
    if previous exists and
       previous.action == requested and
       now >= previous.LastModified + cooldown:
        reject: previous request is final

    seq = 0 if no previous intent, otherwise previous.seq + 1
    attest {bucket, seq, action, next_pcr0}
    write to the key for seq

finalise migration:
    winner = resolve winning intent

    require winner.action == requested
    require now >= winner.LastModified + cooldown
    proceed

Update

After discussing with @aruokhai, it is decided that the trade off of potentially bricking the migration path by enforcing no-forks from predecessors is undesirable.

Instead, it is expected that the operator will make the migration bucket publicly readable to allow independent external verification and monitoring of intents for every PCR0 in the chain.

If the bucket is not public, then users should be wary of entering the system. If the operator later removes public read access then users need to consider the risk of a hidden fork and potentially exit the system. If there is such a hidden fork, users will still have at least the remaining cooldown period from the point of detection to enact the exit.