feat: support local Wasm via EnvoyProxy wasmModules allowlist by daanvinken · Pull Request #9530 · envoyproxy/gateway

daanvinken · 2026-07-20T12:30:50Z

Description

Adds local Wasm modules registered on EnvoyProxy, so Envoy can load
files from the proxy filesystem without the control-plane fetch/cache
path used by HTTP and Image sources.

Design matches #9448 and review discussion (DynamicModule-style allowlist):

Register modules on EnvoyProxy.spec.wasmModules with
source.type: Local and source.local.path
Reference them from EnvoyExtensionPolicy by omitting code and
setting wasm[].name to the registered module name (no path on the policy)
Translator emits AsyncDataSource.local for Local modules
HTTP/Image behavior is unchanged (code still required for those sources)
Control-plane wasm_cluster is only created when a remote Wasm source is present

The Wasm code type enum is now HTTP and Image.
The unused ConfigMap enum value (never implemented) was removed.

Remote/OCI registration on EnvoyProxy is intentionally not implemented yet;
the source shape leaves room for that later.

Fixes #9448

Testing

Unit test for local vs remote Wasm code source in the xDS translator
Translator fixtures for name-only Wasm (registered module resolves to a path;
unknown module name is rejected)
xDS fixture emits code.local.filename and no unused wasm_cluster
CEL cases for omitted code plus wasmModules Local source
e2e: custom Envoy image with the example Wasm baked in
(examples/wasm-module-test), asserts x-wasm-custom: FOO

netlify · 2026-07-20T12:30:56Z

✅ Deploy Preview for cerulean-figolla-1f9435 ready!

Name	Link
🔨 Latest commit	`0418f5e`
🔍 Latest deploy log	https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6aada47b530b2d000847a96e
😎 Deploy Preview	https://deploy-preview-9530--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

daanvinken · 2026-07-20T12:31:10Z

@zhaohuabing if you have some time 🙏

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53e53febd3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-20T12:38:01Z

+			wasms        []ir.Wasm
+			wasmFailOpen bool
+		)
+		if wasms, wasmError, wasmFailOpen = t.buildWasms(policy, resources, gtwCtx.envoyProxy); wasmError != nil {


Scope filesystem Wasm errors per ancestor

When an HTTPRoute is attached to multiple managed Gateways, this call now resolves the filesystem module against each Gateway's EnvoyProxy but appends any failure to one errs value returned to processEnvoyExtensionPolicyForRoute, which then sets the translation error on all ancestor refs. If one Gateway allowlists the module and another does not, the successful ancestor is still marked translation-failed even though its routes get the filter; please report/apply the error per parentRef instead of aggregating it across all ancestors.

Useful? React with 👍 / 👎.

Pre-existing bug with ancestor refs. This should be fixed but is separate from your changes.

codecov · 2026-07-20T12:54:40Z

Codecov Report

❌ Patch coverage is 89.94083% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.35%. Comparing base (3ec9a01) to head (0418f5e).

Files with missing lines	Patch %	Lines
internal/gatewayapi/envoyextensionpolicy.go	89.28%	12 Missing ⚠️
internal/provider/kubernetes/indexers.go	50.00%	3 Missing ⚠️
internal/provider/kubernetes/controller.go	50.00%	1 Missing ⚠️
internal/xds/translator/wasm.go	97.29%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9530      +/-   ##
==========================================
+ Coverage   81.31%   81.35%   +0.03%     
==========================================
  Files         264      264              
  Lines       40994    41044      +50     
==========================================
+ Hits        33336    33391      +55     
+ Misses       7657     7652       -5     
  Partials        1        1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

zhaohuabing · 2026-07-20T14:56:43Z

Unnecessary wasm_cluster emitted for filesystem-only configs (minor, worth fixing)

containsWasm (internal/xds/translator/globalresources.go:144) returns true for any Wasm, so createWasmHTTPServiceCluster still emits the wasm_cluster (the EG HTTP serving cluster, plus its upstream TLS transportSocket) even when every Wasm is a Filesystem source that never contacts it. This is visible in the PR's own fixture wasm-filesystem.clusters.yaml, which contains a full wasm_cluster that no filter references.

Suggest gating on the presence of a remote source:

if route.EnvoyExtensions != nil {
    for _, w := range route.EnvoyExtensions.Wasms {
        if w.Code != nil { // HTTP/Image only
            return true
        }
    }
}

This avoids shipping a dead cluster (and its client-cert secret dependency) to proxies that only use filesystem modules — which is precisely the "skip the control-plane path" benefit the PR advertises.

buildWasms moved inside the per-parentRef loop — cache side effect now repeats (minor)

In translateEnvoyExtensionPolicyForRoute, buildWasms moved from once-per-policy to once-per-parentRef (necessary, since the filesystem path resolves against gtwCtx.envoyProxy, which can differ per Gateway). This matches the existing buildLuas/buildDynamicModules placement, so structurally it's fine.

The wrinkle unique to Wasm: for HTTP/Image sources this now calls t.WasmCache.Get() N times for a route with N parentRefs, where it previously ran once. For IfNotPresent this is a cheap cache hit, but with pullPolicy: Always it can trigger repeated fetches. Not a correctness bug, but a behavioral change worth a comment or a second look — a small memoization keyed by resolved config would eliminate it if it matters.

Enum value ConfigMap silently removed (call out in PR description)

The field-level annotation changed from Enum=HTTP;Image;ConfigMap to Enum=HTTP;Image;Filesystem (wasm_types.go:41). ConfigMap was never implemented (no code path, and the type alias only listed HTTP;Image), so this is stale cleanup rather than a real removal. Still, it's technically a narrowing of the accepted API surface — worth an explicit mention in the PR description/release note in case anyone had it set (it would have been non-functional regardless).

Register Wasm modules on EnvoyProxy.spec.wasmModules and reference them by name from EnvoyExtensionPolicy with type Filesystem. Envoy loads the local file via AsyncDataSource.local and skips the control-plane fetch/cache path used by HTTP and Image sources. Includes generated CRD/helm fixtures, API docs, and the jwt DnsCluster fixture update needed for gen-check. Fixes envoyproxy#9448 Signed-off-by: daanvinken <daanvinken@tythus.com>

Only create the control-plane wasm HTTP service cluster (and gate the client certificate) when a route uses a remote HTTP/Image code source. Filesystem modules load a local path and never use that cluster. Signed-off-by: daanvinken <daanvinken@tythus.com>

Filesystem resolution is Gateway-scoped, so buildWasms runs per parentRef like Lua and DynamicModules. HTTP/Image may call WasmCache.Get once per parent; IfNotPresent hits the cache, Always may re-fetch. Signed-off-by: daanvinken <daanvinken@tythus.com>

ConfigMap was never implemented; the code source enum is now HTTP, Image, and Filesystem only. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken · 2026-07-20T15:22:06Z

Super helpful, thanks @zhaohuabing

I've addressed your comments. I'll see when I can carve out some time to deploy this on an actual cluster and test.
Feel free to leave any comments in the meantime.

zhaohuabing · 2026-07-20T15:24:52Z

Should we mirror DynamicModuleSource to leave room for defining HTTP/OCI WASM extensions in the EnvoyProxy(not now)? cc @envoyproxy/gateway-maintainers

type WasmModuleEntry struct {
    Name   string           `json:"name"`
    Source WasmModuleSource `json:"source"`
}

type WasmModuleSource struct {
    // +kubebuilder:default=Local
    // +unionDiscriminator
    Type  *WasmModuleSourceType  `json:"type,omitempty"`   // Local (+ Remote/Image later)
    Local *LocalWasmModuleSource `json:"local,omitempty"`  // { path }
}

wasmModules:
- name: security-filter
  source:
    type: Local          # defaulted; can be omitted
    local:
      path: /var/lib/envoy/security-filter.wasm

const EnvoyProxyModuleWasmCodeSourceType WasmCodeSourceType = "EnvoyProxyModule"

type EnvoyProxyModuleWasmCodeSource struct {
    // Name references a module in EnvoyProxy.spec.wasmModules.
    Name string `json:"name"`
}

wasm:
- code:
    type: EnvoyProxyModule
    envoyProxyModule:
      name: security-filter        # local or remote — resolved from EnvoyProxy

zhaohuabing · 2026-07-20T15:42:02Z

I'll see when I can carve out some time to deploy this on an actual cluster and test

You should be able to add an e2e test for this, following the dynamic module example. You can use this WASM file. for the test.

jukie · 2026-07-21T03:59:51Z

for defining HTTP/OCI WASM extensions in the EnvoyProxy(not now)?

I agree with the suggested shape for filesystem WASM modules but long-term handling of image/http sources might get awkward so we should discuss that as well.
It doesn't need to be wired up here but maybe we should deprecate the existing image/http source fields and offer operators a RuntimeFlag to control whether EnvoyProxy registration is required or optional?

Register wasmModules with source.type Local and source.local.path, and reference them from EnvoyExtensionPolicy as type EnvoyProxyModule. Leaves room for additional EnvoyProxy module sources later without implementing HTTP/OCI registration yet. Signed-off-by: daanvinken <daanvinken@tythus.com>

Add a custom Envoy image with the example Wasm module baked in, wire it into EXAMPLE_APPS, and add an e2e that registers the module on EnvoyProxy and asserts the filter response header. Signed-off-by: daanvinken <daanvinken@tythus.com>

The example Wasm filter appends text to the response body, which breaks JSON extraction of request properties. Match the HTTP Wasm e2e workaround. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken · 2026-07-21T19:59:51Z

Ready for another review, @zhaohuabing
E2E failures seem unrelated?

zhaohuabing · 2026-07-22T02:54:42Z

Ready for another review, @zhaohuabing E2E failures seem unrelated?

Some of the tests are flaky. You can comment /retest to run them again.

zhaohuabing · 2026-07-22T02:56:17Z

Hi @daanvinken Thanks for working on this. I'll bring up the API design at this week's community meeting so we can reach a decision and unblock you.

jukie · 2026-07-22T04:00:47Z

Pending discussion at community sync but LGTM overall, thanks!

daanvinken · 2026-07-22T09:47:09Z

/retest

Keep wasm-module-test in EXAMPLE_APPS using the new multiline list. Signed-off-by: daanvinken <daanvinken@tythus.com>

Upstream now emits rds.configSource.initialFetchTimeout: 0s. Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing · 2026-08-28T07:50:47Z

code.type: EnvoyProxy + code.envoyProxy.name works, but it's a union member intended to outlive the other two. If we later decide to deprecate the inline HTTP/Image sources and remove them eventually, code collapses to a single-member union.

Should we reference the module from the existing wasm[].name instead, the way dynamicModules[].nam already references the EnvoyProxy registry?

spec:
  wasm:
  - name: auth        # matches EnvoyProxy.spec.wasmModules[].name
    config: {…}

with code taking precedence while it still exists, and a registry lookup by name when it's absent.

cc @guydc @arkodg do we want to deprecate the inline sources and remove them eventually?

daanvinken · 2026-09-07T14:15:21Z

Thanks @zhaohuabing.

wasm[].name already means the filter identity. It is optional. Existing
HTTP/Image policies set it (for example name: wasm-filter) and Envoy
Gateway generates a name when it is omitted. dynamicModule[].name was
always the registry key, so the same field cannot do both jobs without a
new rule.

Two options:

Keep code.type: EnvoyProxy + code.envoyProxy.name in this PR. HTTP/Image stay inline. If those sources later move onto EnvoyProxy.spec.wasmModules, drop code in a later deprecation. The union does not have to collapse here.
Make code optional now. When code is set, HTTP/Image stay as they are. When code is omitted, look up wasm[].name in EnvoyProxy.spec.wasmModules and reject if it is missing. Drop the EnvoyProxy union member. That matches DynamicModule for the registry path and avoids a one-member union later.

I would not reuse wasm[].name as the registry key while code is
still required. That silently changes an existing field.

I'm leaning towards 2. I would wait for @guydc / @arkodg on whether inline HTTP/Image should be deprecated.

guydc · 2026-09-08T17:08:00Z

+1 to inline deprecation.

zhaohuabing · 2026-09-09T07:50:52Z

@daanvinken

Agreed, option 2 is what I had in mind. With code optional, the name-only entry mirrors dynamicModules and there's no one-member union to clean up later. Since @guydc is +1 on deprecating inline sources, I think this unblocks the PR.

When EnvoyExtensionPolicy.wasm[].code is omitted, wasm[].name is looked up in EnvoyProxy.spec.wasmModules. Drop the EnvoyProxy code source union member. HTTP and Image stay on code. Signed-off-by: daanvinken <daanvinken@tythus.com>

Add MaxLength=253 on Wasm.Name so the omitted-code matches() rule stays under the CRD CEL budget. Remove initialFetchTimeout from the wasm-filesystem fixture; main no longer emits it. Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing · 2026-09-15T04:25:13Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9d522ab3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing · 2026-09-17T09:54:19Z

+	//
+	// +kubebuilder:validation:MinLength=1
+	// +kubebuilder:validation:MaxLength=4096
+	// +kubebuilder:validation:Pattern=`^/.*`


Non-blocking nit: LocalDynamicModuleSource.Path documents the same absolute-path requirement but only validates length, while other API path fields generally use only MinLength. Should we add ^/.* consistently? This can be tracked as a separate API cleanup.

Sounds good, I'll track seperately.

Other comments are addressed, thanks again

zhaohuabing

LGTM. Thanks!

The helper only matches remote HTTP/Image Wasm, not name-only local modules. Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing · 2026-09-21T14:15:48Z

/retest

daanvinken requested a review from a team as a code owner July 20, 2026 12:30

chatgpt-codex-connector Bot reviewed Jul 20, 2026

View reviewed changes

daanvinken force-pushed the feat/wasm-filesystem-source-9448 branch from 53e53fe to 089e532 Compare July 20, 2026 12:43

daanvinken force-pushed the feat/wasm-filesystem-source-9448 branch 2 times, most recently from f4fe92e to fddff27 Compare July 20, 2026 14:21

daanvinken force-pushed the feat/wasm-filesystem-source-9448 branch from fddff27 to 3e206c5 Compare July 20, 2026 14:56

daanvinken added 3 commits July 20, 2026 17:17

docs: note ConfigMap Wasm enum value removal in release note

edb360c

ConfigMap was never implemented; the code source enum is now HTTP, Image, and Filesystem only. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken added 3 commits July 21, 2026 15:37

test(e2e): cover EnvoyProxyModule local Wasm load

cea6794

Add a custom Envoy image with the example Wasm module baked in, wire it into EXAMPLE_APPS, and add an e2e that registers the module on EnvoyProxy and asserts the filter response header. Signed-off-by: daanvinken <daanvinken@tythus.com>

test(e2e): skip request body checks for EnvoyProxyModule Wasm

550e57a

The example Wasm filter appends text to the response body, which breaks JSON extraction of request properties. Match the HTTP Wasm e2e workaround. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken changed the title ~~feat: support Filesystem Wasm code source via EnvoyProxy allowlist~~ feat: support EnvoyProxyModule Wasm code source via EnvoyProxy allowlist Jul 21, 2026

Merge branch 'main' into feat/wasm-filesystem-source-9448

bf94ea5

Merge branch 'main' into feat/wasm-filesystem-source-9448

5362fe8

jukie reviewed Jul 24, 2026

View reviewed changes

Comment thread api/v1alpha1/wasm_types.go Outdated

daanvinken added 2 commits August 20, 2026 15:13

Merge upstream/main into feat/wasm-filesystem-source-9448

89cc74e

Keep wasm-module-test in EXAMPLE_APPS using the new multiline list. Signed-off-by: daanvinken <daanvinken@tythus.com>

test: add initialFetchTimeout to wasm-filesystem xDS fixture

b1262db

Upstream now emits rds.configSource.initialFetchTimeout: 0s. Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing added this to the v1.10.0-rc.1 Release milestone Aug 28, 2026

zhaohuabing requested review from a team, arkodg and guydc August 28, 2026 08:18

api: look up local Wasm by name when code is omitted

9c535fb

When EnvoyExtensionPolicy.wasm[].code is omitted, wasm[].name is looked up in EnvoyProxy.spec.wasmModules. Drop the EnvoyProxy code source union member. HTTP and Image stay on code. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken changed the title ~~feat: support EnvoyProxyModule Wasm code source via EnvoyProxy allowlist~~ feat: support local Wasm via EnvoyProxy wasmModules allowlist Sep 9, 2026

daanvinken added 2 commits September 9, 2026 13:59

Merge branch 'main' into feat/wasm-filesystem-source-9448

d35e777

chatgpt-codex-connector Bot reviewed Sep 15, 2026

View reviewed changes

Comment thread release-notes/current/new_features/9448-wasm-filesystem-source.md

docs: note Wasm.Code pointer and name length as breaking

92383b8

Signed-off-by: daanvinken <daanvinken@tythus.com>

zhaohuabing reviewed Sep 17, 2026

View reviewed changes

zhaohuabing mentioned this pull request Sep 18, 2026

feat: adding support for envoy wasm local code mount #9779

Open

11 tasks

zhaohuabing reviewed Sep 18, 2026

View reviewed changes

Comment thread internal/gatewayapi/globalresources.go Outdated

zhaohuabing reviewed Sep 18, 2026

View reviewed changes

Comment thread internal/xds/translator/globalresources.go Outdated

zhaohuabing previously approved these changes Sep 18, 2026

View reviewed changes

zhaohuabing requested a review from a team September 18, 2026 02:41

refactor: rename containsWasm to containsRemoteWasms

9544860

The helper only matches remote HTTP/Image Wasm, not name-only local modules. Signed-off-by: daanvinken <daanvinken@tythus.com>

daanvinken dismissed zhaohuabing’s stale review via 9544860 September 18, 2026 17:33

daanvinken mentioned this pull request Sep 18, 2026

Align absolute-path validation for local module Path fields #10054

Open

Merge branch 'main' into feat/wasm-filesystem-source-9448

0418f5e

zhaohuabing approved these changes Sep 21, 2026

View reviewed changes

zhaohuabing requested a review from a team September 22, 2026 09:35

Conversation

daanvinken commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Uh oh!

netlify Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for cerulean-figolla-1f9435 ready!

Uh oh!

daanvinken commented Jul 20, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

jukie Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zhaohuabing commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daanvinken commented Jul 20, 2026

Uh oh!

zhaohuabing commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhaohuabing commented Jul 20, 2026

Uh oh!

jukie commented Jul 21, 2026

Uh oh!

daanvinken commented Jul 21, 2026

Uh oh!

zhaohuabing commented Jul 22, 2026

Uh oh!

zhaohuabing commented Jul 22, 2026

Uh oh!

jukie commented Jul 22, 2026

Uh oh!

daanvinken commented Jul 22, 2026

Uh oh!

Uh oh!

zhaohuabing commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daanvinken commented Sep 7, 2026

Uh oh!

guydc commented Sep 8, 2026

Uh oh!

zhaohuabing commented Sep 9, 2026

Uh oh!

zhaohuabing commented Sep 15, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

zhaohuabing Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

daanvinken Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

daanvinken Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zhaohuabing left a comment

Choose a reason for hiding this comment

Uh oh!

zhaohuabing commented Sep 21, 2026

daanvinken commented Jul 20, 2026 •

edited

Loading

netlify Bot commented Jul 20, 2026 •

edited

Loading

codecov Bot commented Jul 20, 2026 •

edited

Loading

zhaohuabing commented Jul 20, 2026 •

edited

Loading

zhaohuabing commented Jul 20, 2026 •

edited

Loading

zhaohuabing commented Aug 28, 2026 •

edited

Loading