fix(daily-cache): stop a frozen under-read from outranking the sources on disk - #1218
fix(daily-cache): stop a frozen under-read from outranking the sources on disk#1218therickfactr wants to merge 1 commit into
Conversation
|
Both Local gate for the record — |
efa0618 to
f2e8213
Compare
|
Retitled to the That makes the two CI runs a clean A/B on identical content, and they failed differently each time:
Same tree, disjoint failures — both are flakes, neither is this change. In the second run the parallel suite itself passed whole (261 files, 2 skipped, my new suite included) and the job died in the serial lock step: This PR touches |
…s on disk (getagentseal#1217) A day is derived into the durable cache once and then frozen behind the watermark, while isPartialSurvival lets a fresh derivation SHRINK a day inside the settle window. One parse that missed sources therefore became permanent: the Overview headline sat 27% below the Daily Activity rows printed underneath it, on days whose transcripts were intact the whole time. The read path already parses the dates it is reporting on, so let it reconcile: per (date, provider) keep whichever derivation explains more calls. A cached day whose transcripts have expired still wins, since nothing live can outbid it, and an under-read cached row stops hiding evidence that is sitting on disk. Ties go to the cache, so a re-pricing still lands through the write path only. The write path is untouched: what to freeze stays its decision, and a still-settling day is still defined by its fresh parse. Fixes getagentseal#1217
f2e8213 to
2ca65fc
Compare
Summary
lastComputedDate, whileisPartialSurvivaldeliberately lets a fresh derivation shrink a day inside the settle window. Together, one parse that missed sources becomes permanent — my7 DaysOverview headline read 27% below the sum of the Daily Activity rows printed directly under it (27,346 calls vs 32,608), on five days whose transcripts were intact the whole time.(date, provider)and keeps whichever explains more calls. Carry-forward is untouched — nothing live can outbid a day with no surviving source — and ties go to the cache, so a re-pricing still lands only through the write path.Fixes #1217.
Testing
npm testpasses — 1 failed | 3,500 passed, and that one failure iscli-budget > keeps overview budget lines only on unfiltered overviews, which fails identically onmain(1 failed | 3,494 passed). It is the pre-noon-UTC seeding flake test(budget): seed current-month spend in the past, not at noon UTC #1216 fixes; CI here ran at 08:05 UTC and hit the same one. Zero new failures.npm run buildsucceedsAlso run:
npx tsc --noEmitclean, andnpm run test:locksmatchingmain(1 failed | 35 passed, the knowncache-refresh-lockheartbeat flake, same test both sides).Against real data, on the machine that hit this: the headline now matches a fresh parse day for day, and the
includes $… preserved from expired session logsfootnote — which had been naming exactly the five corrupted days — drops to $0.00.Detail
Evidence it is the cache, not the parser.
daily-cache.v17.json, still on disk from before the upgrade, holds the correct figures (2026-08-26: 2,429 calls; the v29 row says 892). A single-day parse and a 6-month parse of those dates both return the full numbers today. The under-read itself was transient and I could not reproduce it — that limit is written up in #1217. This PR is about the other half: that a transient under-read becomes permanent state with no path back.Why not fix it on the write side. I first tried re-deriving the unsettled tail on each run (
gapStart = min(watermark + 1, today - SETTLE_DAYS)). That breaks six tests which deliberately guarantee the opposite —does not recompute yesterday after it has already been cached,trusts a stamped watermark over an idle tail — no re-derive treadmill, and four more. Those tests are right, so I left them alone; I flagged the change of approach on the issue before rewriting.Cost. No new parse and no new file read.
buildDurablePeriodaggregates a parse it already performed,buildDurableOverviewFromNormalizedIndexreusesnormalizedDaysit already computes, and when cache and parse agree the merge finds nothing to swap. A partial index can never lower a total under this rule, since a thinner slice never wins — which matters for the progressive-startup path, where the index is deliberately incomplete early.Implementation. The reconcile is
mergeDayEntries's existing partial-survival machinery under a new'prefer-richer'mode, so slice/day totals stay reconciled by the code that already does that.tests/durable-underread-reconcile.test.tsfails 4/6 againstmainand passes 6/6 here. It builds a real session fixture on a past date plus an under-reading cache row for it, then asserts the durable headline matches the live parse — along with the three cases that must not change: a genuinely expired day still carries, a richer cached slice is not dragged down to a partial live one, and an equal-call re-pricing leaves the durable value alone.Note for anyone already affected: this stops the symptom wherever a live parse happens, but a cache that already froze an under-read stays wrong on disk. Setting
"complete": falsein~/.cache/codeburn/daily-cache.v<N>.jsonand running once repairs it (verified: all five days healed exactly). Deleting the file instead is not equivalent —MIN_SUPPORTED_VERSIONmakes older generations unadoptable, so a delete drops every carried day whose sources have since expired.