Skip to content

fix(crt): honor wildcard nonProxyHosts entries in CRT-based clients - #7249

Open
zoewangg wants to merge 2 commits into
masterfrom
zoewang/crt-nonproxyhosts-wildcard
Open

fix(crt): honor wildcard nonProxyHosts entries in CRT-based clients#7249
zoewangg wants to merge 2 commits into
masterfrom
zoewang/crt-nonproxyhosts-wildcard

Conversation

@zoewangg

@zoewangg zoewangg commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Motivation and Context

nonProxyHosts lets a client bypass the proxy for hosts a customer lists (via http.nonProxyHosts, the no_proxy environment variable, or the client builder). On CRT-based clients, wildcard entries were not applied: a *.example.com entry or a bare * did not match, so a request to a host the entry was meant to cover connected through the proxy rather than directly.

This affects the CRT-based clients — AwsCrtHttpClient, AwsCrtAsyncHttpClient, and the S3/CRT client — when a proxy is configured and a wildcard nonProxyHosts/no_proxy entry is used. Exact-host and CIDR entries were already applied correctly and are unchanged. The behavior is pre-existing (not a regression). Because a matching host was routed through the proxy instead of connecting directly, traffic a customer intended to keep off the proxy could pass through it; the fix makes wildcard entries behave the same way they do on the other clients.

Root cause. There is one shared nonProxyHosts parser and several downstream matchers with different expectations. The shared parser (SdkHttpUtils) rewrites each glob token by replacing * with the Java-regex fragment .*?. That is correct for the Netty/Apache/url-connection clients, which match with host.matches(regex). The CRT client hands the same set to the native aws-c-http matcher via HttpProxyOptions.setNoProxyHosts; that matcher is curl-style (exact host, dot-anchored suffix, CIDR, bare *) and cannot interpret a Java regex. So *.example.com reached CRT as .*?.example.com, which the native matcher matches against no host, and bare * reached it as .*?, likewise matching nothing.

Modifications

Fix scoped to the CRT path only; the Java-regex clients (Netty, Apache, url-connection) are not modified.

  • Source raw tokens for the CRT path. ProxyConfigProvider gains a rawNonProxyHosts() accessor that returns the split, lowercased tokens WITHOUT the * -> .*? rewrite. It is a default that throws UnsupportedOperationException (not abstract) to preserve backward compatibility for this @SdkProtectedApi interface; the two concrete providers (ProxySystemPropertyConfigProvider, ProxyEnvironmentVariableConfigProvider) override it, delegating the shared split to a new @SdkInternalApi ProxyNonProxyHostParser helper.
  • Translate at the CRT. CrtProxyConfiguration sources rawNonProxyHosts(), and CrtConfigurationUtils.resolveProxy translates each token to the curl form before calling setNoProxyHosts: a leading-* wildcard (*.example.com) becomes the dot-anchored suffix (.example.com); a bare *, exact host names, and CIDR ranges pass through unchanged. Builder-supplied values were already raw, so both CRT input paths (system property / environment variable, and the client builder) are normalized to the same form before translation.
  • Javadoc. The CRT builder nonProxyHosts(Set) / addNonProxyHost(String) docs (crt-core CrtProxyConfiguration.Builder and the aws-crt-client ProxyConfiguration.Builder override) now describe the accepted forms in plain language and note the whitespace requirement.

Supported forms (same as the http.nonProxyHosts system property): an exact host name such as example.com; a leading-* wildcard such as *.example.com; a single * for all hosts; a CIDR range such as 10.0.0.0/8. Entries must not carry surrounding whitespace: a leading or trailing space is treated as part of the host and prevents matching (this affects the common comma-space no_proxy=a.com, *.foo.com spelling and is documented rather than silently trimmed, since the shared parser feeds the untouched Java-regex clients too).

Testing

  • End-to-end mock-proxy routing tests (CrtProxyWildcardNonProxyHostsTest, aws-crt-client): a WireMock proxy observes the routing decision via its request journal (no live destination needed).
  • crt-core unit tests (CrtConnectionUtilsTest): parameterized glob-to-curl translation for the builder and system-property paths, multi-entry mixed sets, both-input-paths-converge, and the null-token filter.
  • **utils unit tests

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@zoewangg
zoewangg requested a review from a team as a code owner August 10, 2026 20:54
@zoewangg
zoewangg requested a review from RanVaknin August 10, 2026 20:59
@zoewangg
zoewangg force-pushed the zoewang/crt-nonproxyhosts-wildcard branch from 88feb25 to a1cee23 Compare August 10, 2026 21:34
CRT-based clients silently ignored wildcard nonProxyHosts entries: the
shared parser rewrites glob * to the Java-regex .*?, which the native
curl-style matcher cannot read, so *.example.com and bare * matched no
host and those hosts were routed through the proxy.

Source the raw glob tokens for the CRT path and translate them to the
curl dot-suffix form at the CRT sink; the Java-regex clients (Netty,
Apache, url-connection) are untouched.
@zoewangg
zoewangg force-pushed the zoewang/crt-nonproxyhosts-wildcard branch from a1cee23 to e2f8c52 Compare August 10, 2026 21:53
EventLoopGroup.closeStaticDefault();
HostResolver.closeStaticDefault();
}

@RanVaknin RanVaknin Aug 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test that asserts the proxy routing behavior when no_proxy is supplied with comma space arguments?

}

@ParameterizedTest(name = "builder nonProxyHost \"{0}\" -> curl \"{1}\"")
@CsvSource({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this support IPv6 inputs?

CRT-based clients support only an exact host, a leading *. suffix
wildcard, a bare *, or a CIDR range in nonProxyHosts. A wildcard in any
other position (e.g. 192.168.*, internal*) is not matched by the native
matcher, so the host is routed through the proxy. Log this at WARN
during proxy resolution instead of failing silently.
@zoewangg
zoewangg force-pushed the zoewang/crt-nonproxyhosts-wildcard branch from cc1dd5c to 8bcead0 Compare August 12, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants