Skip to content

feat(sync): add kenari sync adapter - #3171

Open
doedja wants to merge 5 commits into
anomalyco:devfrom
doedja:feat/kenari-sync-adapter
Open

feat(sync): add kenari sync adapter#3171
doedja wants to merge 5 commits into
anomalyco:devfrom
doedja:feat/kenari-sync-adapter

Conversation

@doedja

@doedja doedja commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Adds Kenari (kenari.id) to the hourly model sync workflow so its catalog stays current automatically instead of via hand-opened PRs.

How it works

  • Fetches the public https://kenari.id/v1/models endpoint. No API key and no repo secret needed.
  • Cost stays 0 by policy: Kenari bills in IDR from a prepaid wallet at its own catalog prices, so USD figures here would be a misleading currency conversion. The zero cost matches every existing hand-authored providers/kenari entry.
  • Existing factored TOMLs are preserved exactly as authored. Limits keep inheriting the canonical base model.
  • New models sync only when a canonical models/ metadata entry exists to inherit capability facts from (factorBaseModel). Kenari's dash IDs resolve to the dotted canon generically (grok-4-5 -> xai/grok-4.5), no per-model alias table.
  • Models with no canonical entry are skipped and surfaced in the sync-report notice rather than authored from API data that has no name/release-date fields.
  • Reasoning effort values come verbatim from the endpoint's reasoning_options array (per-model, e.g. "reasoning_options":["low","medium","high","xhigh","max"] on claude-fable-5). A model with "reasoning": true and no advertised efforts gets reasoning_options = [], matching the existing hand-authored entries.

Scope

This PR is the adapter only: packages/core/src/sync/index.ts, packages/core/src/sync/providers/kenari.ts, and its test. No TOMLs are added, changed, or deleted here. An earlier revision of this description carried a "first sync (included in this PR)" table, which was a dry-run result rather than a file list. Removed, since nothing in the diff writes a model file.

Where the catalog stands today

Measured 2026-08-10 against the live endpoint and this repo's default branch:

count
Models the endpoint serves 50
providers/kenari/models/*.toml in this repo 38
Served but not in the repo 15
In the repo but no longer served 3 (grok-build-0-1, kimi-k2-6:free, kimi-k2-7-code:free)

The 15 is an upper bound on creations, not a prediction: any of them without a canonical models/ entry is skipped and reported instead. Most of that drift is also covered by #4393, so whichever of the two lands first shrinks the other's first run.

Reasoning effort values advertised by the endpoint

Re-checked against live on 2026-08-10, all still current:

Model /v1/models reasoning_options
claude-fable-5, claude-sonnet-5 ["low","medium","high","xhigh","max"]
gpt-5-6-luna, gpt-5-6-sol, gpt-5-6-terra ["none","low","medium","high","xhigh","max"]
grok-4-5 ["low","medium","high"]
mimo-v2-5:free none advertised (reasoning: true) -> reasoning_options = []

One thing worth disclosing about the effort values

Until 2026-08-10 the Kenari endpoint derived each model's reasoning_options from the openrouter slice of this repo's own feed, not from the model author. Had this adapter run before that date, it would have round-tripped models.dev data back into models.dev as though it were an independent observation, and any error in the OpenRouter slice would have been laundered into the providers/kenari rows as corroboration.

That is fixed at the source. The endpoint now reads the model author's own first-party slice when the author is a provider key this repo already carries, and falls back to the OpenRouter list otherwise. deepseek-v4-flash is the clearest case: it advertised ["high","xhigh"] (OpenRouter's list) and now advertises ["low","high","max"], which is what DeepSeek's own slice in this repo says. #4393 hand-corrects the rows that carried the old values.

So the adapter now reads a value the host derives from the author, and for a model whose author is not in this repo it still reads a value that traces back to the OpenRouter slice. That is the same provenance the hand-authored rows already had, and it is worth knowing rather than assuming every synced value is independent.

Verification

Current as of 2026-08-10:

  • Both CI checks (review, validate) green on this branch
  • The endpoint values in the table above re-measured against live

From when the adapter was written, against the catalog as it stood then:

  • bun models:sync kenari twice: second run is a no-op (0 created, 0 updated, 0 deleted)
  • bun validate passes
  • bun run test: same 3 pre-existing failures as a clean checkout of dev, none related

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [low] [violation] providers/kenari/models/claude-fable-5.toml:5 (also claude-sonnet-5.toml, gpt-5-6-luna.toml/sol/terra, grok-4-5.toml) - Check: The audit-reasoning-options skill requires explicit provider evidence for non-standard effort values (none, xhigh, max), and PR-body citations must state what each source proves. Why: The PR body cites the https://kenari.id/v1/models endpoint but never mentions reasoning_options at all, while the new files assert provider-specific effort sets like ["low","medium","high","xhigh","max"] (Claude 5), ["none","low","medium","high","xhigh","max"] (GPT-5.6), and ["low","medium","high"] (Grok 4.5). The adapter code sources these from the endpoint's reasoning_options field, but that mapping is not documented in the PR body, so the capability claims cannot be reviewed against the cited source without it. Action: Add a sentence to the PR body stating that the /v1/models response's reasoning_options array is the source for each new model's effort values (ideally noting the exact values the endpoint advertises per model), so the non-standard values are directly mapped to the cited endpoint.

@doedja

doedja commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the action item: the PR body now states that effort values are sourced verbatim from the endpoint's reasoning_options array, with a per-model table of the exact values https://kenari.id/v1/models advertises today (re-checked against the live endpoint before posting). Models advertising reasoning: true with no effort list get reasoning_options = [], same as the existing hand-authored entries.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@doedja

doedja commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Opened #3380 as a data-only refresh of the kenari catalog, generated by running this adapter against the current dev branch. The live endpoint has drifted since this PR's snapshot (18 models added, 3 removed), which is the drift this adapter is meant to absorb hourly.

If #3380 lands first I will rebase this PR so its first sync becomes a no-op and it reduces to the adapter code plus the idempotence proof. Happy to also fit it into the labeling scheme from #3370 (model-sync, provider:kenari) if that is the direction sync PRs are heading.

rekram1-node pushed a commit that referenced this pull request Jul 22, 2026
Adds 18 models and removes 3 no longer served, generated by running the
kenari sync adapter (PR #3171) against the current dev branch. Cost stays
0 by policy (IDR prepaid wallet), reasoning_options come verbatim from
the endpoint.
jernejstrasner pushed a commit to s-ference/models.dev that referenced this pull request Aug 8, 2026
Adds 18 models and removes 3 no longer served, generated by running the
kenari sync adapter (PR anomalyco#3171) against the current dev branch. Cost stays
0 by policy (IDR prepaid wallet), reasoning_options come verbatim from
the endpoint.
doedja added a commit to doedja/models.dev that referenced this pull request Aug 8, 2026
Data-only sync against https://kenari.id/v1/models. Cost stays 0 per the
IDR-wallet billing policy.

Added (10): claude-opus-5, claude-sonnet-4-6, gemini-3-6-flash,
minimax-m2-7, minimax-m2-7-highspeed, nemotron-3-ultra-550b-a55b:free,
qwen3-7-flash, qwen3-8-max, step-3-7-flash, step-3-7-flash:free.

Removed (3): grok-build-0-1, kimi-k2-6:free, kimi-k2-7-code:free.

Skipped (4): laguna-s-2-1:free, longcat-2-0:free, mimo-v2-5-pro-ultraspeed,
north-mini-code:free. No canonical models/ metadata exists for these IDs
yet; hand-author when an upstream entry lands.

The kenari sync adapter (anomalyco#3171) was used to generate this catalog locally
and will rebase onto this branch separately.
Rebases anomalyco#3171 onto current upstream/dev (458519b). Original data-file
changes dropped: they were 30 days stale and the catalog refresh lives
in chore/kenari-catalog-refresh-aug, which this PR is meant to follow.

Adapter code only:

- Public /v1/models endpoint, no API key or repo secret needed.
- Cost stays 0 by policy: Kenari bills in IDR from a wallet at its own
  catalog prices, so USD figures would be a misleading conversion.
- Existing factored TOMLs are preserved as authored; limits keep
  inheriting the canonical base model (Kenari's context figures are
  themselves ingested from models.dev, so they are not authoritative).
- New models sync only when a canonical models/ metadata entry exists
  (dash IDs resolve to dotted canon, e.g. grok-4-5 -> xai/grok-4.5);
  unresolvable models are skipped with a sync-report notice.

Idempotence proof: with this commit on top of chore/kenari-catalog-refresh-aug,
bun models:sync kenari produces 0 created / 0 updated / 0 removed / 45 unchanged
across two consecutive runs. bun validate exits 0.

Structural adaptation from the original PR: index.ts gained several new
providers (hyper, merge-gateway, nano-gpt, ofox, cortecs, requesty,
tinfoil, baseten) and aggregators group became a multi-line array with
merge-gateway/nano-gpt/ofox/requesty slots. The kenari import + provider
registration + aggregators entry were re-inserted alphabetically.
@doedja
doedja force-pushed the feat/kenari-sync-adapter branch from 8a18661 to d0168e9 Compare August 9, 2026 03:28
@doedja

doedja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current dev, and this is now pure code: 2 files, the adapter plus its registration, with zero data files in the branch.

I opened #4393 as the data-only refresh, generated by running this adapter on current dev. Once that lands, this adapter's first sync is a no-op, which was the thing I said I would do here rather than leaving you to review code and a data dump together.

The idempotence is measured, not assumed: running the adapter twice against the refreshed data gives 10 created, 0 updated, 3 removed on the first pass and 0 created, 0 updated, 0 deleted on the second, with validate.ts exiting 0.

One thing worth flagging rather than hiding: the live endpoint lists 50 models and the synced set is 45. The 5 without an entry have no canonical models/ base to inherit from, so the adapter skips them and reports them through skippedNotice. They are listed in #4393. Happy to hand-author those canonical entries separately if you want them covered.

Also still happy to take the model-sync and provider:kenari labels from #3370 whenever that scheme is ready.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] packages/core/src/sync/providers/kenari.ts:157 - Check: Relay reasoning_options must not invent “no control” from missing source data; preserve authored controls when the host omits them. Why: reasoningOptions() returns [] when reasoning === true and reasoning_options is omitted or empty, and the caller uses reasoningOptions(model) ?? existing.reasoning_options. Because [] is defined, the ?? fallback never runs, so any hand-authored efforts (e.g. DeepSeek high/xhigh, Gemini/Nemotron graded lists) are wiped whenever the endpoint leaves efforts unset. Peer adapters (OpenRouter, NanoGPT, Kilo) preserve existing options when the source omits controls; [] means affirmative no caller control, not uncertainty. Action: Return undefined when the API omits reasoning_options so existing entries are preserved; only write [] when the API explicitly returns an empty list (or for brand-new models with no prior entry via the runner’s preserve path).
  • [medium] [possible mistake] packages/core/src/sync/providers/kenari.ts:148 - Check: New Kenari models must resolve base_model reliably to canonical models/ metadata. Why: resolveKenariBaseModel only tries resolveCanonicalBaseModel(owned_by + id|dotted id). That requires owned_by to be a CANONICAL_PROVIDER_PREFIXES key (no zhipuai, bare-lab typos, etc.) and never falls back to bare unique filename resolution. Requesty/NanoGPT/EmpirioLabs use resolveModelMetadataBaseModel or ID-derived candidates so bare aggregator IDs still factor. Mis-tagged or unknown owned_by silently skips creatable models even when models/<lab>/<id>.toml exists. Action: After the owned_by attempts, fall back to resolveModelMetadataBaseModel on the raw and dash-to-dot IDs (and add any Kenari owned_by aliases you observe, e.g. zhipuai → zai/zhipuai).
  • [medium] [violation] packages/core/src/sync/providers/kenari.ts:165 - Check: Sync code must locate models/ independently of process cwd (see other providers’ MODELS_DIR). Why: canonicalName() reads path.join("models", …) from the working directory. Other adapters resolve metadata via import.meta.dirname. If sync is not launched from the repo root, :free display names fail open (freeName returns undefined) and new free variants lose the "(Free)" name override. Action: Resolve the metadata root from import.meta.dirname (same pattern as openrouter.ts / requesty.ts) and read canonical names from that absolute base.
  • [medium] [possible mistake] packages/core/src/sync/providers/kenari.ts:1 - Check: New sync adapters with custom ID normalization and reasoning mapping need focused regression tests (sync.md / existing sync.test.ts coverage for EmpirioLabs, LLM Gateway, Hyper, etc.). Why: Dash↔dot resolution, :free handling, existing-vs-new factoring, and the omitted-vs-empty reasoning behavior are easy to regress and are untested in this PR. Action: Add unit tests for resolveKenariBaseModel / buildKenariModel covering dotted canon IDs, :free naming, preservation of authored reasoning_options when the API omits efforts, and zero-cost factored output.

…no controls

An omitted list meant the adapter emitted [], which is an affirmative
no-control assertion. Because [] is not nullish it defeated the
?? existing.reasoning_options fallback, so a hand-authored effort list was
wiped whenever the endpoint left efforts unset. Only an explicitly empty
list from the API carries that meaning now.

Adds three regression tests: preservation on omission, an explicit empty
list, and a published list.
@doedja

doedja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

All four review items are fixed, each with a test that fails when the fix is reverted.

  • reasoningOptions() returns undefined when the endpoint omits efforts, so both ?? existing sites and the runner's preserveReasoningOptions work. Only an explicitly empty list writes [].
  • resolveKenariBaseModel falls back to resolveModelMetadataBaseModel on the bare and dotted IDs when owned_by is not a canonical prefix. It still skips when the match is ambiguous.
  • canonicalName reads from a MODELS_DIR derived from import.meta.dirname, matching openrouter and requesty, so :free names no longer depend on cwd.
  • reasoning_options parses as strings and filters to the catalog effort set, deduped, so one unknown value no longer throws and kills the whole sync. A list with no known values is treated as silence rather than as "no control".

Also dropped modalities and tool_call from the schema, which were declared but never read.

9 tests added. Full sync.test.ts is 122 pass, 1 fail, and that failure (DeepInfra preserves live modalities for new base models) also fails on clean dev. tsc --noEmit reports 36 errors on this branch and 36 on clean dev, none in kenari files.

The base-resolution fix changed the data, so I regenerated #4393: it now covers 48 of the 50 live models instead of 45. ling-3-0-flash:free and north-mini-code:free still skip, since neither has canonical metadata to inherit from.

Happy to take the model-sync and provider:kenari labels from #3370 whenever that scheme is ready.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] packages/core/src/sync/providers/kenari.ts:148 - Check: New models must resolve a canonical models/ entry reliably (same-surface peers use bare-ID fallbacks). Why: resolveKenariBaseModel only tries resolveCanonicalBaseModel("${owned_by}/${id}") (plus dash→dot). That requires Kenari’s owned_by to be an OpenRouter-style lab prefix (openai, x-ai, z-ai, …). Values like system, kenari, or zhipuai never match, so every create is skipped. Idempotence on existing TOMLs only exercises the existing.base_model update path, and there is no test for the create/resolve path—so a broken owned_by mapping can ship unnoticed. Action: After the prefixed attempts, fall back to resolveModelMetadataBaseModel on the bare/dotted id (as cortecs/hyper/requesty do); add unit tests for representative Kenari ids/owned_by pairs; confirm against live /v1/models that new rows actually resolve.
  • [medium] [violation] packages/core/src/sync/providers/kenari.ts:104 - Check: Sync must stay resilient when the provider API adds fields outside the catalog enum. Why: reasoning_options: z.array(EffortValue) makes parseModels throw on any unknown effort string, failing the entire Kenari hourly job instead of syncing the rest of the catalog. Peer adapters (crossmodel, hyper, empiriolabs) accept strings and filter to known efforts. Action: Parse efforts as z.array(z.string()) (or similar) and keep only values in the catalog effort set when building reasoning_options.
  • [low] [possible mistake] packages/core/src/sync/providers/kenari.ts:165 - Check: Provider modules must resolve models/ via a stable path (not process cwd). Why: canonicalName reads path.join("models", …), unlike every other adapter’s import.meta.dirname → repo models/ root. Free-tier display names ("${base} (Free)") silently drop if cwd is not the repo root. Action: Use the shared MODELS_DIR / modelMetadata() pattern and cover freeName in tests.

doedja added 2 commits August 9, 2026 12:01
…from the source file

Two follow-ups from review.

resolveKenariBaseModel only tried the canonical provider prefix, so a model
whose owned_by is a lab name we do not alias was skipped even when
models/<lab>/<id>.toml existed. It now falls back to resolving the bare ID
against the metadata filenames, which strips :free and takes only an
unambiguous single match, so a wrong guess stays a skip.

canonicalName read models/ relative to the working directory, so :free
display names silently lost their (Free) suffix whenever the sync ran from
anywhere but the repo root. The root now comes from import.meta.dirname,
matching openrouter and requesty.

Adds three more tests: dotted-ID resolution, resolution with an unusable
owned_by, and :free naming with zero-cost factored output.
…to what is read

Parsing reasoning_options as an enum meant one unrecognised effort string
threw in parseModels and failed the whole hourly sync instead of the single
field. Efforts are parsed as strings now and filtered to the catalog set,
deduped. A list whose values are all unknown is treated as silence rather
than as an assertion that the control is gone.

Also drops modalities and tool_call from the schema, which were declared but
never read (capability facts come from the canonical base, and passthrough
keeps the raw payload), and passes limit straight to factorBaseModel instead
of an all-undefined object that resolved to the same thing.

Adds three tests: an unknown effort parses, unknown values are dropped while
known ones survive, and an all-unknown list keeps the authored efforts.
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Checking one more interaction: whether the create path and preserveReasoningOptions force [], and whether the factored path drops provider/experimental.

A canonical filename can carry a version the host's ID drops. Kenari serves
`north-mini-code:free` while the lab entry is
`models/cohere/north-mini-code-1-0.toml`. Every resolution path compares
filenames exactly (`resolveCanonicalBaseModel` builds `${candidate}.toml`,
`resolveModelMetadataBaseModel` falls back to an exact ID then an exact
basename), and `canonicalCandidates` rewrites only `-fast` plus a few
lab-specific shapes, so nothing bridges that gap.

The model was therefore skipped as "no canonical metadata entry" while its
metadata sat in the tree the whole time. That is the failure mode this adapter
exists to remove: the skip is silent, it reads as missing lab data, and the
answer looks like a hand-authored PR per model.

Adds a last-resort pass that matches a bare ID against canonical entries whose
name is that ID plus a trailing release marker, and takes it ONLY when exactly
one exists anywhere in the tree. Stems with siblings (`mistral-large`, `gpt-4o`,
`claude-opus`) stay unresolved, because choosing a release on the host's behalf
is a guess and a wrong `base_model` is worse than a visible skip. The marker
pattern is digits only, so a word suffix like `-code` or `-flash` can never
collapse two different models. The index is built once per process.

This runs after every existing path, so no ID that resolves today changes.

nano-gpt solves the same case with a hand-maintained BASE_MODEL_ALIASES entry
(`cohere/north-mini-code`). That works, but it needs an edit per model forever,
which is the opposite of what an hourly unattended sync is for. Happy to switch
to an alias table if you would rather keep the mechanism uniform across
adapters.

Two tests, both break-tested: the north-mini case was red before the fix, and
relaxing the single-match guard to `>= 1` turns the ambiguity test red, so the
guard is doing real work rather than passing by luck.

`bun test` is unchanged at 4 pre-existing failures (open-weight metadata
weights links, DeepInfra modalities, and two snapshot tests), verified against a
clean checkout of this branch before and after.
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Checking how resolved existing models expose base_model and whether the adapter mishandles it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant