Follow-up to #138 (feat(runtime): correct enclave clock drift via hypervisor PTP clock).
Summary
#138 correctly established the architecture for synchronizing the
enclave's CLOCK_REALTIME against the hypervisor PTP clock
(/dev/ptp0): perform an initial hard step at startup, then
continuously discipline the clock.
The remaining issue is the control algorithm.
The current implementation periodically corrects accumulated phase
error using ADJ_OFFSET_SINGLESHOT, but never estimates or compensates
for frequency error. As a result, each synchronization cycle reduces
the accumulated clock offset, but the underlying clock rate remains
unchanged and the same offset immediately begins accumulating again.
This proposal replaces the current loop with a frequency-disciplined
servo using:
- atomic PTP measurements,
- rolling estimation of phase and frequency,
- continuous frequency correction,
- explicit uncertainty bounds.
Motivation
The enclave clock is used for several security-sensitive operations,
including:
- freshness anchors,
- S3 Object Lock
retain-until,
- PCR0 lineage,
- TLS certificate validity,
- K/V TTLs,
- stream IDs,
- migration cooldown.
For these operations we want the clock to converge toward the PTP
reference rather than periodically correcting accumulated offset.
Issues
1. Phase-only controller
runClockSync periodically measures the offset between
CLOCK_REALTIME and the PTP reference.
When the measured offset exceeds the configured threshold it applies
ADJ_OFFSET_SINGLESHOT, but it never estimates or compensates for clock
frequency error.
With the current parameters:
| Quantity |
Value |
| Native drift |
~11.6 ppm |
| Synchronization interval |
5 min |
| Offset accumulated per interval |
~3.5 ms |
Each synchronization cycle reduces accumulated offset but does not
eliminate its cause. The controller bounds clock offset but never
converges toward zero steady-state error.
2. Non-atomic offset measurement
Offset is currently computed using two sequential clock reads:
ClockGettime(phc)
ClockGettime(CLOCK_REALTIME)
This introduces a systematic bias equal to the latency of the PHC read.
The kernel already provides atomic cross timestamps via
PTP_SYS_OFFSET_PRECISE, with PTP_SYS_OFFSET_EXTENDED as a fallback.
We should use those interfaces instead.
3. Single-sample estimator
Each synchronization cycle computes its correction from a single offset
measurement.
Measurement noise, hypercall latency and scheduling delay therefore feed
directly into the controller.
A rolling estimator with outlier rejection simultaneously estimates:
- phase error,
- frequency error,
- measurement uncertainty.
This is the same class of controller used by chrony and linuxptp.
4. /dev/ptp0 should be mandatory
Gracefully degrading when /dev/ptp0 is unavailable is appropriate for
the TCG test runner.
On production Nitro hardware it silently reintroduces the free-running
clock bug fixed by #138.
Failure to initialize or discipline /dev/ptp0 should therefore be fatal.
Why this matters
Each enclave disciplines its clock independently.
Two enclaves running identical software on different hosts can therefore
accumulate different clock offsets and disagree by several milliseconds
while both consider themselves synchronized.
For comparison, chrony routinely achieves sub-microsecond
synchronization against the same ptp_kvm source.
The remaining gap is therefore dominated by the control algorithm rather
than the underlying clock source.
Proposed Fix
Replace the current phase-only controller with a
frequency-disciplined servo while preserving the architecture introduced
in #138.
-
Measure offset using PTP_SYS_OFFSET_PRECISE, falling back to
PTP_SYS_OFFSET_EXTENDED.
-
Estimate phase, frequency and uncertainty using a rolling regression
window with outlier rejection.
-
Apply ADJ_FREQUENCY together with ADJ_OFFSET via
ClockAdjtime(ADJ_NANO) to continuously compensate for clock rate
error.
-
Treat /dev/ptp0 as mandatory in production enclaves.
The following remain unchanged:
Follow-up to #138 (
feat(runtime): correct enclave clock drift via hypervisor PTP clock).Summary
#138 correctly established the architecture for synchronizing the
enclave's
CLOCK_REALTIMEagainst the hypervisor PTP clock(
/dev/ptp0): perform an initial hard step at startup, thencontinuously discipline the clock.
The remaining issue is the control algorithm.
The current implementation periodically corrects accumulated phase
error using
ADJ_OFFSET_SINGLESHOT, but never estimates or compensatesfor frequency error. As a result, each synchronization cycle reduces
the accumulated clock offset, but the underlying clock rate remains
unchanged and the same offset immediately begins accumulating again.
This proposal replaces the current loop with a frequency-disciplined
servo using:
Motivation
The enclave clock is used for several security-sensitive operations,
including:
retain-until,For these operations we want the clock to converge toward the PTP
reference rather than periodically correcting accumulated offset.
Issues
1. Phase-only controller
runClockSyncperiodically measures the offset betweenCLOCK_REALTIMEand the PTP reference.When the measured offset exceeds the configured threshold it applies
ADJ_OFFSET_SINGLESHOT, but it never estimates or compensates for clockfrequency error.
With the current parameters:
Each synchronization cycle reduces accumulated offset but does not
eliminate its cause. The controller bounds clock offset but never
converges toward zero steady-state error.
2. Non-atomic offset measurement
Offset is currently computed using two sequential clock reads:
This introduces a systematic bias equal to the latency of the PHC read.
The kernel already provides atomic cross timestamps via
PTP_SYS_OFFSET_PRECISE, withPTP_SYS_OFFSET_EXTENDEDas a fallback.We should use those interfaces instead.
3. Single-sample estimator
Each synchronization cycle computes its correction from a single offset
measurement.
Measurement noise, hypercall latency and scheduling delay therefore feed
directly into the controller.
A rolling estimator with outlier rejection simultaneously estimates:
This is the same class of controller used by chrony and linuxptp.
4.
/dev/ptp0should be mandatoryGracefully degrading when
/dev/ptp0is unavailable is appropriate forthe TCG test runner.
On production Nitro hardware it silently reintroduces the free-running
clock bug fixed by #138.
Failure to initialize or discipline
/dev/ptp0should therefore be fatal.Why this matters
Each enclave disciplines its clock independently.
Two enclaves running identical software on different hosts can therefore
accumulate different clock offsets and disagree by several milliseconds
while both consider themselves synchronized.
For comparison, chrony routinely achieves sub-microsecond
synchronization against the same
ptp_kvmsource.The remaining gap is therefore dominated by the control algorithm rather
than the underlying clock source.
Proposed Fix
Replace the current phase-only controller with a
frequency-disciplined servo while preserving the architecture introduced
in #138.
Measure offset using
PTP_SYS_OFFSET_PRECISE, falling back toPTP_SYS_OFFSET_EXTENDED.Estimate phase, frequency and uncertainty using a rolling regression
window with outlier rejection.
Apply
ADJ_FREQUENCYtogether withADJ_OFFSETviaClockAdjtime(ADJ_NANO)to continuously compensate for clock rateerror.
Treat
/dev/ptp0as mandatory in production enclaves.The following remain unchanged:
ptp_kvm,chronydinside the enclave.