Skip to content

fix(daily-cache): stop a frozen under-read from outranking the sources on disk - #1218

Open
therickfactr wants to merge 1 commit into
getagentseal:mainfrom
therickfactr:fix/daily-cache-settle-window
Open

fix(daily-cache): stop a frozen under-read from outranking the sources on disk#1218
therickfactr wants to merge 1 commit into
getagentseal:mainfrom
therickfactr:fix/daily-cache-settle-window

Conversation

@therickfactr

@therickfactr therickfactr commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • A day is derived into the durable cache once and then frozen behind lastComputedDate, while isPartialSurvival deliberately lets a fresh derivation shrink a day inside the settle window. Together, one parse that missed sources becomes permanent — my 7 Days Overview headline read 27% below the sum of the Daily Activity rows printed directly under it (27,346 calls vs 32,608), on five days whose transcripts were intact the whole time.
  • The read path already parses the dates it reports on, so it now reconciles the two derivations per (date, provider) and keeps whichever explains more calls. Carry-forward is untouched — nothing live can outbid a day with no surviving source — and ties go to the cache, so a re-pricing still lands only through the write path.
  • The write path is unchanged: what to freeze stays its decision, a still-settling day is still defined by its fresh parse, and nothing re-parses.

Fixes #1217.

Testing

  • I have tested this locally against real data (not just unit tests)
  • npm test passes — 1 failed | 3,500 passed, and that one failure is cli-budget > keeps overview budget lines only on unfiltered overviews, which fails identically on main (1 failed | 3,494 passed). It is the pre-noon-UTC seeding flake test(budget): seed current-month spend in the past, not at noon UTC #1216 fixes; CI here ran at 08:05 UTC and hit the same one. Zero new failures.
  • npm run build succeeds

Also run: npx tsc --noEmit clean, and npm run test:locks matching main (1 failed | 35 passed, the known cache-refresh-lock heartbeat flake, same test both sides).

Against real data, on the machine that hit this: the headline now matches a fresh parse day for day, and the includes $… preserved from expired session logs footnote — which had been naming exactly the five corrupted days — drops to $0.00.

before   $6,415.71   27,346 calls    includes $4,108.78 preserved from expired session logs
after    $8,880.38   34,527 calls    (no carried footnote)

Detail

Evidence it is the cache, not the parser. daily-cache.v17.json, still on disk from before the upgrade, holds the correct figures (2026-08-26: 2,429 calls; the v29 row says 892). A single-day parse and a 6-month parse of those dates both return the full numbers today. The under-read itself was transient and I could not reproduce it — that limit is written up in #1217. This PR is about the other half: that a transient under-read becomes permanent state with no path back.

Why not fix it on the write side. I first tried re-deriving the unsettled tail on each run (gapStart = min(watermark + 1, today - SETTLE_DAYS)). That breaks six tests which deliberately guarantee the opposite — does not recompute yesterday after it has already been cached, trusts a stamped watermark over an idle tail — no re-derive treadmill, and four more. Those tests are right, so I left them alone; I flagged the change of approach on the issue before rewriting.

Cost. No new parse and no new file read. buildDurablePeriod aggregates a parse it already performed, buildDurableOverviewFromNormalizedIndex reuses normalizedDays it already computes, and when cache and parse agree the merge finds nothing to swap. A partial index can never lower a total under this rule, since a thinner slice never wins — which matters for the progressive-startup path, where the index is deliberately incomplete early.

Implementation. The reconcile is mergeDayEntries's existing partial-survival machinery under a new 'prefer-richer' mode, so slice/day totals stay reconciled by the code that already does that.

tests/durable-underread-reconcile.test.ts fails 4/6 against main and passes 6/6 here. It builds a real session fixture on a past date plus an under-reading cache row for it, then asserts the durable headline matches the live parse — along with the three cases that must not change: a genuinely expired day still carries, a richer cached slice is not dragged down to a partial live one, and an equal-call re-pricing leaves the durable value alone.

Note for anyone already affected: this stops the symptom wherever a live parse happens, but a cache that already froze an under-read stays wrong on disk. Setting "complete": false in ~/.cache/codeburn/daily-cache.v<N>.json and running once repairs it (verified: all five days healed exactly). Deleting the file instead is not equivalent — MIN_SUPPORTED_VERSION makes older generations unadoptable, so a delete drops every carried day whose sources have since expired.

@therickfactr

Copy link
Copy Markdown
Contributor Author

Both test legs are red on tests/cli-budget.test.ts > keeps overview budget lines only on unfiltered overviewsexpected 'CodeBurn September 2026\n\nNo usage …' to contain 'Monthly budget:'. That is the pre-noon-UTC seeding flake #1216 describes, not this change: CI ran at 08:05 UTC, inside the window, and it is the same single failure I get on main locally. Everything else in both legs passes (258/259 files), the new suite included.

test (22.13.0) · test (22)

Local gate for the record — npm test on main: 1 failed | 3,494 passed; on this branch: 1 failed | 3,500 passed, same test. Happy to rebase once #1216 lands so this goes fully green.

@therickfactr therickfactr changed the title Stop a frozen under-read day from suppressing the sources on disk fix(daily-cache): stop a frozen under-read from outranking the sources on disk Sep 1, 2026
@therickfactr
therickfactr force-pushed the fix/daily-cache-settle-window branch from efa0618 to f2e8213 Compare September 1, 2026 15:19
@therickfactr

Copy link
Copy Markdown
Contributor Author

Retitled to the fix(daily-cache): convention and rewrote the body against .github/PULL_REQUEST_TEMPLATE.md. The force-push carried no code changeefa06184 and f2e82139 have the same tree, b3715ed389a99657f222c696c1ac07fb57a4e5a9, so only the commit subject moved.

That makes the two CI runs a clean A/B on identical content, and they failed differently each time:

run cli-budget Cache-lock suite (serial)
08:05 UTC fail (pre-noon-UTC seeding, #1216) pass
15:19 UTC pass timed out at 5m

Same tree, disjoint failures — both are flakes, neither is this change. In the second run the parallel suite itself passed whole (261 files, 2 skipped, my new suite included) and the job died in the serial lock step: The action 'Cache-lock suite (serial)' has timed out after 5 minutes. That is the fence never loses to its own heartbeat (in-process serialization) burning its 10 retries — it takes ~330s on my machine against main too, which is already over the step's 5-minute budget. Looks like the #904 / #1196 neighbourhood.

This PR touches src/daily-cache.ts, src/usage-aggregator.ts and one new test; the lock path is src/parser.ts / src/session-cache.ts, untouched. I don't have re-run rights here — a maintainer re-running test (22) should settle it.

…s on disk (getagentseal#1217)

A day is derived into the durable cache once and then frozen behind the
watermark, while isPartialSurvival lets a fresh derivation SHRINK a day
inside the settle window. One parse that missed sources therefore became
permanent: the Overview headline sat 27% below the Daily Activity rows
printed underneath it, on days whose transcripts were intact the whole
time.

The read path already parses the dates it is reporting on, so let it
reconcile: per (date, provider) keep whichever derivation explains more
calls. A cached day whose transcripts have expired still wins, since
nothing live can outbid it, and an under-read cached row stops hiding
evidence that is sitting on disk. Ties go to the cache, so a re-pricing
still lands through the write path only.

The write path is untouched: what to freeze stays its decision, and a
still-settling day is still defined by its fresh parse.

Fixes getagentseal#1217
@therickfactr
therickfactr force-pushed the fix/daily-cache-settle-window branch from f2e8213 to 2ca65fc Compare September 2, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: an under-read day inside the settle window is frozen forever — Overview headline stays below the sum of its own Daily Activity rows

1 participant