Use a frozen reference report to make changes inspectable

Give the reference a precise role

The frozen reference edition is a stable teaching artifact built from original synthetic inputs. It supplies an expected result against which calculation or presentation changes can be reviewed. It is not a current health statement about a live model and should not acquire a fresh operational timestamp merely because somebody opens the page.

The interactive report accepts local page inputs and recalculates their diagnostics. It can therefore show a different result without changing the frozen edition. A user exploring the drift preset is changing the scenario, not receiving a new observation from a production system. The distinction should remain visible in headings, downloads and report labels.

Preserve the identities needed for comparison

A replayable package needs the full input, engine version, model identity, model version, as-of timestamp and output. Baseline values, edges, smoothing and policy belong in that package because they can change a score without changing current observations. A screenshot or a short list of metrics is insufficient to reconstruct those choices.

The site exports full input-and-report JSON and a metrics CSV. The JSON is the fuller replay artifact. The CSV is a convenient table of results, not a complete specification of the baseline, expected dates or policy. A reviewer should not assume that importing the metrics table can reproduce a report from its outputs alone.

Work through two different kinds of change

In a synthetic scenario edit, keep the engine version fixed and move current feature counts from 4, 6, 6, 4 to 2, 4, 6, 8. The report’s PSI changes because the input changes. The frozen reference remains the same and provides a useful point of comparison.

In a hypothetical implementation correction, keep the entire input fixed and change a defective boundary rule to the documented left-inclusive convention. Values exactly on edges may move bins, changing the output. That comparison needs a new engine version and a release note explaining the corrected behavior. It must not be described as a new market event.

Use provenance without overstating authenticity

W3C’s provenance model distinguishes entities, activities and responsible agents. A compact report package can apply that conceptual separation: input artifact, calculation activity and resulting report. This publication does not claim formal PROV conformance, and a file hash identifies bytes rather than proving that a source observation was economically accurate.

Record a digest when exact artifact identity matters, but retain a readable explanation as well. Two differently formatted JSON files can have different byte hashes while expressing the same values. Conversely, a correct hash of a flawed input faithfully identifies the flawed input. Integrity and substantive correctness require different checks.

Make supersession visible

If an error affects the reference edition, preserve the earlier artifact and identify the corrected edition rather than silently rewriting history. State which calculations or descriptions changed and whether prior conclusions are affected. W3C’s web-data best practices provide useful primary guidance on version information and provenance, while the specific publication workflow remains this site’s own design.

A reader should be able to navigate between the interactive report, frozen reference and change register and understand their different purposes. None is a recurring monitor or an approval certificate. Their value is a transparent, inspectable example of how monitoring evidence can remain coherent as scenarios and software evolve.

Keep in the decision record

• A frozen teaching edition is not a live status endpoint.
• Full input-and-report JSON supports replay; metrics CSV alone does not.
• Distinguish scenario edits from implementation corrections.

Source notes

1. W3C — PROV-O, Recommendation, 30 April 2013
2. W3C — Data on the Web Best Practices, Recommendation, 31 January 2017
3. Google SRE — Release Engineering

Sources inform the stated methods and governance context. All example monitoring observations are synthetic. No regulatory approval or production monitoring certification is implied.

Continue the analysis