SEN-16-007 CONTROLLED Experiment summary

Experiment Delta: Holdover Beyond Design

A reference was deliberately cut off from every external input and left to run for 400 days to find out how it would fail. It failed in the way that is hardest to detect.

Dated
March 22, 2016
Classification
CONTROLLED
Directorate
Directorate I
Station
ARGENT
Custody
MERIDIAN
Status
PUBLISHED

CONTROLLED. Published with editorial redaction. Withheld passages concern methods or third parties. Ensemble composition and the holdover algorithm's parameters are withheld; both are current design detail in Project Polaris.

SEN-16-007 CONTROLLED Experiment summary PUBLISHED COPY Disclosed 2016-03-22

Question

How does a timing reference fail when it has been isolated for longer than anyone designed it to be isolated?

Not whether it degrades — that is known and modelled. The question is the shape of the failure, because SEN-93-112 established that the institution’s exposure comes from failures that are confident rather than failures that are large.

Method

An ensemble at ARGENT was isolated from all external reference on 4 January 2015 and operated for 400 days with no correction, no comparison and no external input of any kind.

A parallel ensemble, identically configured, remained disciplined normally. Neither ensemble’s operators had access to the other’s data. Comparison was performed once, at the end, by a third party.

Result

Degradation over the first ~180 days was close to the model. This was expected and is not interesting.

Beyond roughly day 200 the behaviour departed from the model in a specific and awkward way: the ensemble’s internal agreement improved while its accuracy continued to degrade.

The members converged on each other. Their mutual consistency — the quantity every practical health metric is built on — rose steadily, while the ensemble as a whole drifted further from truth.

By day 400 the ensemble was reporting excellent health by every internal measure available to it, and was wrong by a margin that would have been operationally significant.

Why this matters

An isolated ensemble has no access to truth. It can only measure agreement. When common-mode drift dominates, agreement stops being evidence of accuracy and becomes evidence of shared error — and the system’s confidence rises at precisely the moment its correctness collapses.

This is the failure signature the institution most fears: not an instrument that breaks, but an instrument that becomes serenely, internally consistent and wrong. It is the same shape as the 1993 incident, arrived at from the opposite direction.

Consequences for Polaris

  1. Internal agreement is not a health metric past a defined isolation horizon, and Polaris explicitly discounts it.
  2. Confidence must decay with time since external reference, monotonically, regardless of what the ensemble reports about itself. A Polaris timestamp carries its own age and refuses to present as fresh.
  3. Disagreement is preserved rather than averaged, so a later reviewer can reconstruct what the ensemble knew and when.
  4. Holdover specifications are stated with an explicit validity horizon beyond which the institution declines to make a claim at all.

Note on the parallel ensemble

The disciplined ensemble behaved correctly throughout and is unremarkable.

It is mentioned because the experiment’s most useful property was that neither team could see the other. The isolated ensemble’s operators reported high confidence in their instrument at day 380. They were experienced, they were careful, and they were reading the only numbers they had.

MERIDIAN custody · facsimile may differ in pagination · redactions not reversible from this copy

Index terms

  • timing
  • polaris
  • experiment
  • metrology

Related records

Referenced by

Inspect in discovery graph →

Released by the Disclosure Board under Protocol Sigma. Redactions are applied at the section level and are not reversible from the published copy. Requests for review may be sent to [email protected].