Resolve duplicate and unexpected periods before monitoring

Treat period identity as part of the model contract

The report expects one row per declared period. Model identity and version apply to the input as a whole, while date identifies each period within it. This simple arrangement is sufficient for the synthetic monthly examples. A multi-portfolio or multi-instrument system would need richer identities before it could safely aggregate observations.

A duplicate date can mean a repeated delivery, competing correction, accidental concatenation or two distinct entities that were incorrectly assigned the same key. The fixture engine does not guess among these explanations. It reports duplicate dates and withholds drift and complete-window performance so the ambiguity remains visible.

A count match can conceal two defects

Start with twenty expected monthly dates and twenty corresponding rows. Remove June and append a second May row. Observed count remains twenty, yet unique count falls to nineteen. The report must identify both the missing June key and the repeated May key; reporting only one net difference would imply that the input is complete.

If the two May returns are 0.01 and 0.03, keeping the last row chooses a return according to file order. Reversing the file changes the result even though the same evidence is present. A valid correction needs source precedence or a documented revision decision, not whichever row happened to be appended most recently.

Unexpected periods alter the question

Add September to a report contracted for January 2023 through August 2024. The new row might be genuine in a real system, but it lies outside this analysis window. Including it changes the outcome horizon and feature sample. The appropriate action is either to correct the input or to approve a new expected window with its own identity.

Do not silently clip unexpected records simply to recover a metric. Preserve their dates and explain the proposed exclusion. An upstream time-zone conversion, month-label convention or reporting lag may have placed an otherwise relevant observation under the wrong key. A mechanical trim can hide the underlying alignment defect and leave it recurring.

Separate ordering from deduplication

The engine sorts valid outcome rows by date without mutating the input. Sorting makes path-dependent calculations reproducible, but it does not choose among duplicates. A stable order cannot convert ambiguous identity into a trustworthy wealth path, and numerical agreement between two duplicate rows does not prove independent observation.

For a synthetic repair exercise, retain input A with the duplicated May row, input B with the verified correction and a short decision record. Reconcile expected and unique dates again before recalculating returns. If the repair changes the reported outcome, preserve both reports and state that one was superseded because of a data defect.

Design controls around the failure mechanism

A repeated transmission calls for idempotent ingestion; a legitimate revision needs version selection; an identity collision needs a better key. Those are different engineering tasks. W3C provenance provides useful concepts for linking inputs, transformations and outputs, while the specific reconciliation workflow here is an original teaching design rather than a prescribed standard.

In a real managed service, a duplicate alert should link to the affected artifact and ingestion attempt, with a responsible owner and a bounded recovery plan. The browser example does not implement that service. It demonstrates why retaining a clear error is more informative than calculating an attractive metric from a dataset whose period identities have not been resolved.

Keep in the decision record

• Matching row totals can conceal missing and duplicated periods.
• Sorting does not resolve revision precedence.
• Preserve original and corrected inputs with the repair rationale.

Source notes

1. W3C — PROV-O, Recommendation, 30 April 2013
2. W3C — Data Quality Vocabulary, Working Group Note, 15 December 2016
3. W3C — Data on the Web Best Practices, Recommendation, 31 January 2017

Sources inform the stated methods and governance context. All example monitoring observations are synthetic. No regulatory approval or production monitoring certification is implied.

Continue the analysis