-
Notifications
You must be signed in to change notification settings - Fork 26
test(playground): smoke-test the server plugins and showcase the testing kit #542
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+420
−0
Merged
Changes from all commits
Commits
Show all changes
106 commits
Select commit
Hold shift + click to select a range
1b38046
feat(appkit): make PluginContext telemetry injectable
IamGalymzhan dd503f5
feat(appkit): ship @databricks/appkit/testing and migrate first stub
IamGalymzhan 55859c8
test(appkit): migrate route-handler-errors context stub to mockPlugin…
IamGalymzhan 4d37d5e
docs(appkit): document the testing kit and ship a template example test
IamGalymzhan 5821f6b
docs(appkit): fix testing-kit examples to instantiate the plugin class
IamGalymzhan cc790d0
refactor(appkit): tighten FakeToolResponse so a missing value is a ty…
IamGalymzhan bace753
refactor(appkit): make tools/test-helpers a shim over the shipped tes…
IamGalymzhan cfde155
fix(appkit): normalize CRLF in expectStream SSE parsing; sharpen test…
IamGalymzhan de5b40b
fix(appkit): resolve repo-wide Biome error blocking CI
IamGalymzhan 8acf7ee
fix(appkit): address cross-model review findings in the testing kit
IamGalymzhan 9297c73
chore(appkit): drop knip vitest-ignore now that vitest is a real peer…
IamGalymzhan 3f11ea3
fix(appkit): make vitest a normal dependency, not a package-wide peer
IamGalymzhan c2c1aa9
refactor(appkit): rename mockPluginContext to createTestPluginContext
IamGalymzhan 2564971
refactor(appkit): dedupe testing fixtures and tidy test-plugin-context
IamGalymzhan e02375d
Merge branch 'main' into feat/testing-kit
IamGalymzhan 1e28a96
fix(appkit): resolve third-review findings in the testing kit
IamGalymzhan 01dd25c
test(appkit): dogfood the testing kit on analytics and genie plugins
IamGalymzhan f09ca5d
refactor(appkit): address testing-kit review feedback
IamGalymzhan 90288bb
docs(appkit): move the testing guide under Plugins
IamGalymzhan ac0f0cb
test(appkit): fold dogfood tests into plugin suites
IamGalymzhan 1ebdee1
test(appkit): restore toHaveLength(1) on the analytics OBO dispatch test
IamGalymzhan 5cdd207
test(appkit): re-assert genie SSE payloads after the expectStream swap
IamGalymzhan 448947f
fix(appkit): drop fabricated workspace-client fields from createMockR…
IamGalymzhan 45601bf
chore(appkit): merge origin/main and reconcile with the oxlint toolchain
IamGalymzhan 7586c50
feat(appkit): add a never-crash mock WorkspaceClient to the testing kit
IamGalymzhan d4f6355
refactor(appkit): converge the two mock-workspace-client builders
IamGalymzhan e10cb3b
feat(appkit): split the lifecycle exit from the teardown
IamGalymzhan ff574f7
feat(appkit): expose close() on the app handle
IamGalymzhan ef172e1
feat(appkit): make the process-wide singletons re-bootable
IamGalymzhan 3719d5d
feat(appkit): add createTestApp — the customer-grade plugin test harness
IamGalymzhan a524958
feat(appkit): publish the harness surface, add createTestPlugin, and …
IamGalymzhan 28848bd
test(appkit): tighten the mock-client type contract from tarball find…
IamGalymzhan 5bdb52c
fix(appkit): address code-review findings in the testing kit
IamGalymzhan 48b1a4c
docs(appkit): document `close` as a reserved plugin name
IamGalymzhan c2649e6
chore: drop the **/.claude ignores from knip, oxlint, and oxfmt
IamGalymzhan 473ecd0
refactor(appkit): slim the comments added by the testing-kit work
IamGalymzhan efb310d
test(appkit): merge duplicate assertions in the mock-client suite
IamGalymzhan fd1fcd3
test(appkit): compact the testing-kit suites
IamGalymzhan 6a64107
fix(appkit): address four review findings in the testing kit
IamGalymzhan 9adaafd
Merge remote-tracking branch 'origin/main' into feat/testing-kit
IamGalymzhan 1cc97b6
chore: restore the dev-playground client lockfile
IamGalymzhan 2eef458
fix(appkit): correct double singleton release and simplify the testin…
IamGalymzhan 6d7f5d6
chore: drop the .claude ignores and restore the client lockfile
IamGalymzhan f70f082
Merge remote-tracking branch 'origin/main' into split/foundation
IamGalymzhan f509275
Merge branch 'feat/testing-kit' into feat/testing-kit-harness
IamGalymzhan 5a61b29
refactor(appkit): trim the cache reset test and its comments
IamGalymzhan 4bd7e60
docs(appkit): relocate and tighten the app-handle comments
IamGalymzhan 9070c99
test(appkit): cover the singleton release on close()'s timeout path
IamGalymzhan 83f23d5
fix(appkit): lowercase mock request header keys, as Express does
IamGalymzhan 46a44b4
Merge branch 'feat/testing-kit' into feat/testing-kit-harness
IamGalymzhan 7a158f5
fix(appkit): lowercase caller header keys in the harness request methods
IamGalymzhan a8503f5
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan 04839f0
chore(appkit): acknowledge the testing kit's tarball growth in the ba…
IamGalymzhan f08dc07
Revert "chore(appkit): acknowledge the testing kit's tarball growth i…
IamGalymzhan 9875a2c
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan 583b3fe
fix(appkit): address Pawel's review — singleton leak, renames, and gu…
IamGalymzhan b761c75
fix(appkit): match the real user-context shape in the harness stub
IamGalymzhan c538e93
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan 9b51959
fix(appkit): share one user-context fake across the testing kit
IamGalymzhan c82c40e
docs(appkit): deslop the testing guide and fix its sidebar slot
IamGalymzhan 91f5fd0
chore(docs): regenerate the appkit-ui stylesheet
IamGalymzhan b1bf73c
feat(appkit): add strict mode to the mock workspace client
IamGalymzhan 217a816
fix(appkit): address the straightforward half of the second review pass
IamGalymzhan 5d2b638
fix(appkit): close the leftover halves of three review comments
IamGalymzhan b4697c4
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan 01c0bbc
fix(appkit): allow one harness app at a time
IamGalymzhan cbdd4a8
ci(appkit): run the scaffolded template's tests, and ignore its stagi…
IamGalymzhan b3e3211
refactor(appkit): drop the dead re-exports from the deprecated test-h…
IamGalymzhan 9a8be5b
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan fc8a11e
feat(appkit): add withEnv scoped-environment test helper
IamGalymzhan f0cfffb
feat(appkit): add createApiError test-error factory
IamGalymzhan 7c4216c
feat(appkit): add composition options to createTestPluginContext
IamGalymzhan 65c020c
test(appkit): adopt withEnv for inline env handling in plugin suites
IamGalymzhan 86d68c3
docs(appkit): document the new testing-kit helpers
IamGalymzhan 5289f60
refactor(appkit): drop unused import and bindings in the composition …
IamGalymzhan 566fbb1
refactor(appkit): share env capture/restore and guard withEnv error p…
IamGalymzhan 3e6d6e4
Merge branch 'main' into feat/testing-kit-harness
IamGalymzhan 77a74ab
feat(appkit): add useTestCache helper to the testing kit
IamGalymzhan ca368b7
test(appkit): add internal createCacheMock passthrough fake
IamGalymzhan 286a417
test(appkit): migrate analytics suite off the cache mock to useTestCache
IamGalymzhan 40e0a18
test(appkit): migrate metric suite off the cache mock to useTestCache
IamGalymzhan 1d65f38
test(appkit): migrate ai-search suite off the cache mock to useTestCache
IamGalymzhan 62a25c7
test(appkit): adopt shared createCacheMock in files/plugin suite
IamGalymzhan a009492
docs(appkit): document useTestCache for asserting cache behaviour
IamGalymzhan 19e21a7
test(appkit): adopt shared createCacheMock across passthrough cache s…
IamGalymzhan 224675b
fix(appkit): give createCacheMock getOrExecute the real userKey arity
IamGalymzhan b85a422
docs(appkit): tighten testing-kit guide prose
IamGalymzhan 1a5b142
feat(appkit): add useTestApp to wire the harness hooks for a suite
IamGalymzhan fbddbb3
test(appkit): fake the workspace client through the kit, not module m…
IamGalymzhan 0158cde
test(playground): smoke-test the playground's own server plugins
IamGalymzhan 5f35ad8
test(playground): demonstrate each testing-kit helper against real pl…
IamGalymzhan 9648553
chore: gitignore the .codex-tmp agent scratch dir
IamGalymzhan 52d9f06
test(appkit): replace createCacheMock with useTestCache across suites
IamGalymzhan 159f0a1
test(appkit): drop the stale EXPERIMENT label from mkdir test
IamGalymzhan b6a54ad
fix(appkit): make createTestPluginContext options auto-restore actual…
IamGalymzhan 2529902
refactor(appkit): drop inert refcount from singleton-reset machinery
IamGalymzhan 6dffb59
refactor(appkit): drop the multi-boot/embeddable surface from the tes…
IamGalymzhan 4376fcd
Merge remote-tracking branch 'origin/main' into feat/testing-kit-harness
IamGalymzhan 16ab51e
Merge branch 'feat/testing-kit-harness' into feat/testing-kit-helpers
IamGalymzhan 99388b5
Merge branch 'feat/testing-kit-helpers' into feat/dev-playground-smok…
IamGalymzhan 8d5c1db
refactor(appkit): collapse the shutdown/dispose split into shutdown({…
IamGalymzhan d9ecdae
Merge branch 'feat/testing-kit-harness' into feat/testing-kit-helpers
IamGalymzhan b5dff77
Merge branch 'feat/testing-kit-helpers' into feat/dev-playground-smok…
IamGalymzhan eac6bc3
test(playground): strengthen testing-kit integration assertions (review)
IamGalymzhan 2686b36
test(playground): harden smoke assertions from review round 2
IamGalymzhan 0307a33
Merge branch 'main' into feat/dev-playground-smoke-tests
IamGalymzhan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| import { createTestApp } from "@databricks/appkit/testing"; | ||
| import { describe, expect, test, vi } from "vitest"; | ||
|
|
||
| import { lakebaseExamples } from "./lakebase-examples-plugin"; | ||
| import { reconnect } from "./reconnect-plugin"; | ||
| import { telemetryExamples } from "./telemetry-example-plugin"; | ||
|
|
||
| /** | ||
| * Smoke tests for the playground's own server plugins. | ||
| * | ||
| * `tests/` holds Playwright specs that fake `/api` responses at the browser | ||
| * boundary (`page.route` + `fulfill`), so the Express server never runs there. | ||
| * These cover the other side: the plugins boot and answer over real HTTP with | ||
| * the Databricks data plane faked by the harness — no workspace, no | ||
| * credentials, no network. | ||
| */ | ||
|
|
||
| /** | ||
| * Read one SSE payload, then hang up. | ||
| * | ||
| * `expectStream` buffers a source to completion, and the reconnect stream is | ||
| * five messages three seconds apart — so asserting through it would cost ~12s | ||
| * for a smoke test. | ||
| */ | ||
| async function firstSSEPayload(res: Response): Promise<unknown> { | ||
| if (!res.body) throw new Error("expected a streaming body, got none"); | ||
| const reader = res.body.getReader(); | ||
| const decoder = new TextDecoder(); | ||
| let buffered = ""; | ||
| try { | ||
| while (!buffered.includes("\n\n")) { | ||
| const { value, done } = await reader.read(); | ||
| if (done) break; | ||
| buffered += decoder.decode(value, { stream: true }); | ||
| } | ||
| const data = buffered.split("\n").find((line) => line.startsWith("data:")); | ||
| return data ? JSON.parse(data.slice("data:".length).trim()) : undefined; | ||
| } finally { | ||
| await reader.cancel(); | ||
| } | ||
| } | ||
|
|
||
| describe("dev-playground server plugins", () => { | ||
| test("all three boot together and register under their manifest names", async () => { | ||
| await using app = await createTestApp({ | ||
| plugins: [reconnect(), telemetryExamples(), lakebaseExamples()], | ||
| }); | ||
|
|
||
| expect(app.plugins.reconnect).toBeDefined(); | ||
| expect(app.plugins["telemetry-examples"]).toBeDefined(); | ||
| expect(app.plugins["lakebase-examples"]).toBeDefined(); | ||
| }); | ||
|
|
||
| test("GET /api/reconnect answers", async () => { | ||
| await using app = await createTestApp({ plugins: [reconnect()] }); | ||
|
|
||
| const res = await app.get("/api/reconnect"); | ||
|
|
||
| expect(res.status).toBe(200); | ||
| await expect(res.json()).resolves.toEqual({ message: "Reconnected" }); | ||
| }); | ||
|
|
||
| test("the reconnect stream opens as SSE and emits its first message", async () => { | ||
| await using app = await createTestApp({ plugins: [reconnect()] }); | ||
|
|
||
| const res = await app.get("/api/reconnect/stream?sessionId=smoke"); | ||
|
|
||
| expect(res.status).toBe(200); | ||
| expect(res.headers.get("content-type")).toContain("text/event-stream"); | ||
| await expect(firstSSEPayload(res)).resolves.toMatchObject({ | ||
| type: "message", | ||
| count: 1, | ||
| total: 5, | ||
| content: "Message 1 of 5", | ||
| }); | ||
| }); | ||
|
|
||
| test("POST /api/telemetry-examples/combined threads the userId through every span", async () => { | ||
| // This route really calls `fetch("https://example.com")` (its | ||
| // external-api span). Left alone the suite would need the internet, so | ||
| // non-app requests are stubbed — requests to the app under test must pass | ||
| // through, because that is how the harness reaches its own server. Match on | ||
| // the app's *actual* base URL rather than string-matching localhost, so it | ||
| // stays correct whatever host/port the harness binds (IPv4 or IPv6 `[::1]`). | ||
| const realFetch = globalThis.fetch; | ||
| let appBaseUrl: string | undefined; | ||
| vi.stubGlobal("fetch", (input: RequestInfo | URL, init?: RequestInit) => { | ||
| const url = String(input instanceof Request ? input.url : input); | ||
| return appBaseUrl && url.startsWith(appBaseUrl) | ||
| ? realFetch(input, init) | ||
| : Promise.resolve(new Response("stubbed", { status: 200 })); | ||
| }); | ||
|
|
||
| try { | ||
| await using app = await createTestApp({ plugins: [telemetryExamples()] }); | ||
| appBaseUrl = app.baseUrl; | ||
|
|
||
| const res = await app.post("/api/telemetry-examples/combined", { | ||
| body: { userId: "smoke-user" }, | ||
| }); | ||
|
|
||
| expect(res.status).toBe(200); | ||
| // A 200 means the whole nested-span body ran against the real | ||
| // TelemetryProvider: tracer, meter, and logger. | ||
| await expect(res.json()).resolves.toMatchObject({ | ||
| success: true, | ||
| result: { userId: "smoke-user" }, | ||
| }); | ||
| } finally { | ||
| vi.unstubAllGlobals(); | ||
| } | ||
| }); | ||
|
|
||
| test("lakebase-examples registers its routes when Lakebase is configured", async () => { | ||
| // A bare 404 can't tell "plugin degraded" from a path typo, so assert the | ||
| // route is *present*. injectRoutes gates only on PGHOST/LAKEBASE_ENDPOINT, | ||
| // so with those set the example routes register. setup() still fails fast — | ||
| // a ConfigurationError with no PGUSER, before any pool connect, so no | ||
| // network — and the plugin swallows it, so the app boots regardless. | ||
| await using app = await createTestApp({ | ||
| plugins: [lakebaseExamples()], | ||
| env: { | ||
| PGHOST: "test-host.example", | ||
| LAKEBASE_ENDPOINT: "test-endpoint.example", | ||
| }, | ||
| }); | ||
|
|
||
| expect(app.plugins["lakebase-examples"]).toBeDefined(); | ||
| // Present, not 404: the health route answers (503, since the pool never | ||
| // really connected) — proving the route registered, not that a path exists | ||
| // by accident. | ||
| expect( | ||
| (await app.get("/api/lakebase-examples/raw/health")).status, | ||
| ).not.toBe(404); | ||
| }); | ||
|
|
||
| test("lakebase-examples registers no routes when Lakebase is unconfigured", async () => { | ||
| // Its setup() and injectRoutes() both bail on missing PGHOST/LAKEBASE_ENDPOINT. | ||
| // The app must still boot; the routes must simply be absent — the same real | ||
| // route that answers above is 404 here, so presence tracks configuration. | ||
| await using app = await createTestApp({ | ||
| plugins: [lakebaseExamples()], | ||
| env: { PGHOST: "", LAKEBASE_ENDPOINT: "" }, | ||
| }); | ||
|
|
||
| expect(app.plugins["lakebase-examples"]).toBeDefined(); | ||
| expect((await app.get("/api/lakebase-examples/raw/health")).status).toBe( | ||
| 404, | ||
| ); | ||
| }); | ||
| }); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.