Skip to content

Replace the Phase-Only Clock Sync Loop with a Frequency-Disciplined Servo #143

Description

@aruokhai

Follow-up to #138 (feat(runtime): correct enclave clock drift via hypervisor PTP clock).

Summary

#138 correctly established the architecture for synchronizing the
enclave's CLOCK_REALTIME against the hypervisor PTP clock
(/dev/ptp0): perform an initial hard step at startup, then
continuously discipline the clock.

The remaining issue is the control algorithm.

The current implementation periodically corrects accumulated phase
error
using ADJ_OFFSET_SINGLESHOT, but never estimates or compensates
for frequency error. As a result, each synchronization cycle reduces
the accumulated clock offset, but the underlying clock rate remains
unchanged and the same offset immediately begins accumulating again.

This proposal replaces the current loop with a frequency-disciplined
servo using:

  • atomic PTP measurements,
  • rolling estimation of phase and frequency,
  • continuous frequency correction,
  • explicit uncertainty bounds.

Motivation

The enclave clock is used for several security-sensitive operations,
including:

  • freshness anchors,
  • S3 Object Lock retain-until,
  • PCR0 lineage,
  • TLS certificate validity,
  • K/V TTLs,
  • stream IDs,
  • migration cooldown.

For these operations we want the clock to converge toward the PTP
reference rather than periodically correcting accumulated offset.


Issues

1. Phase-only controller

runClockSync periodically measures the offset between
CLOCK_REALTIME and the PTP reference.

When the measured offset exceeds the configured threshold it applies
ADJ_OFFSET_SINGLESHOT, but it never estimates or compensates for clock
frequency error.

With the current parameters:

Quantity Value
Native drift ~11.6 ppm
Synchronization interval 5 min
Offset accumulated per interval ~3.5 ms

Each synchronization cycle reduces accumulated offset but does not
eliminate its cause. The controller bounds clock offset but never
converges toward zero steady-state error.


2. Non-atomic offset measurement

Offset is currently computed using two sequential clock reads:

ClockGettime(phc)
ClockGettime(CLOCK_REALTIME)

This introduces a systematic bias equal to the latency of the PHC read.

The kernel already provides atomic cross timestamps via
PTP_SYS_OFFSET_PRECISE, with PTP_SYS_OFFSET_EXTENDED as a fallback.

We should use those interfaces instead.


3. Single-sample estimator

Each synchronization cycle computes its correction from a single offset
measurement.

Measurement noise, hypercall latency and scheduling delay therefore feed
directly into the controller.

A rolling estimator with outlier rejection simultaneously estimates:

  • phase error,
  • frequency error,
  • measurement uncertainty.

This is the same class of controller used by chrony and linuxptp.


4. /dev/ptp0 should be mandatory

Gracefully degrading when /dev/ptp0 is unavailable is appropriate for
the TCG test runner.

On production Nitro hardware it silently reintroduces the free-running
clock bug fixed by #138.

Failure to initialize or discipline /dev/ptp0 should therefore be fatal.


Why this matters

Each enclave disciplines its clock independently.

Two enclaves running identical software on different hosts can therefore
accumulate different clock offsets and disagree by several milliseconds
while both consider themselves synchronized.

For comparison, chrony routinely achieves sub-microsecond
synchronization against the same ptp_kvm source.

The remaining gap is therefore dominated by the control algorithm rather
than the underlying clock source.


Proposed Fix

Replace the current phase-only controller with a
frequency-disciplined servo while preserving the architecture introduced
in #138.

  1. Measure offset using PTP_SYS_OFFSET_PRECISE, falling back to
    PTP_SYS_OFFSET_EXTENDED.

  2. Estimate phase, frequency and uncertainty using a rolling regression
    window with outlier rejection.

  3. Apply ADJ_FREQUENCY together with ADJ_OFFSET via
    ClockAdjtime(ADJ_NANO) to continuously compensate for clock rate
    error.

  4. Treat /dev/ptp0 as mandatory in production enclaves.

The following remain unchanged:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions