Repository navigation
Conversation
|
/ok to test 3f1fd80 |
Cluster Verification ReportVerified end-to-end on a real multi-tenant OpenShift (ROSA) cluster with two gateways sharing one base domain and wildcard certificate — the exact ambiguous scenario from #4840. Environment
Results
Operator-managed ingress was used per the PR's documented model: exact-host passthrough Routes created per emitted hashed hostname, each targeting its own gateway's Service. Also exercised during setup: Helm rendering of |
… clusters Closes NVIDIA#4840 Add an optional service_routing_gateway_name gateway setting (TOML, CLI, env, Helm) that disambiguates exposed service URLs when multiple gateways share a cluster, ingress, and wildcard certificate. When configured, exposed-service hostnames become a deterministic 16-character lowercase base32 SHA-256 label of (gateway name, workspace, sandbox, service) under the same base domain, keeping every hostname exactly one label deep for wildcard-certificate compatibility. Unset gateways preserve the existing readable hostname format byte-for-byte, and configured gateways still accept legacy hosts during migration. Hashed hosts resolve through an in-memory route index hydrated from persisted ServiceEndpoint records at startup, refreshed on miss for multi-replica deployments, maintained across expose, delete, and sandbox cleanup, and enforced for collisions before endpoint persistence. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
3f1fd80 to
28551bd
Compare
|
Rebased onto the latest @TaylorMutch the branch was force-pushed for the rebase, so the earlier |
|
/ok to test 28551bd |
Summary
Adds an optional
service_routing_gateway_namegateway setting that disambiguates exposed-service URLs when multiple gateways share a Kubernetes cluster, ingress, and wildcard certificate. When configured, service hostnames become a deterministic 16-character lowercase base32 SHA-256 label of(gateway name, workspace, sandbox, service)under the same base domain — exactly one label deep, so a single shared*.domainwildcard certificate keeps working, and each gateway's URLs are globally unique. Unset gateways preserve the existing readable hostname format byte-for-byte.Related Issue
Closes #4840
Changes
crates/openshell-core/src/config.rs:ServiceRoutingConfig.service_routing_gateway_namefield,with_service_routing_gateway_namebuilder, DNS-1123-label validation, and config tests.crates/openshell-server/src/config_file.rs:[openshell.gateway] service_routing_gateway_nameTOML parsing/serialization.crates/openshell-server/src/cli.rs:--service-routing-gateway-name/OPENSHELL_SERVICE_ROUTING_GATEWAY_NAMEwith existing CLI > env > TOML precedence, startup validation, and precedence tests.crates/openshell-server/src/service_routing.rs: SHA-256/NUL-delimited tuple hashing with a hand-rolled fixed-size lowercase base32 encoder (no new dependency), hashedendpoint_hostoutput,ServiceRouteIndex(register/reserve/remove/lookup/rebuild), async state-awareresolve_service_hostsupporting both hashed and legacy hosts, and routing tests with fixed vectors.crates/openshell-server/src/lib.rs: route index inServerState, hydrated from persistedServiceEndpointrecords at startup before listeners bind; ambiguous persisted hashes fail startup.crates/openshell-server/src/http.rs: HTTP middleware and loopback plaintext router use the async state-aware service-host classification.crates/openshell-server/src/grpc/service.rs: hash route reserved before endpoint persistence (AlreadyExistson collision, no write), rolled back on persistence failure, removed after successful delete; collision test.crates/openshell-server/src/compute/mod.rs: sandbox cleanup removes route index entries for deleted endpoints.deploy/helm/openshell/:server.serviceRoutingGatewayNamevalue rendered into gateway TOML via_toml.tpl(empty stays unset; explicitgatewayConfigtakes precedence), chart tests, regenerated README.docs/how-it-works/gateways/configuration.mdx(field reference),docs/how-it-works/gateways/overview.mdx(hostname grammar — also fixes the pre-existing omission of the workspace segment),docs/how-it-works/sandboxes/overview.mdx,docs/kubernetes/ingress.mdx(operator-managed exact-host ingress guidance for multi-gateway shared-cert deployments),docs/kubernetes/setup.mdx.Deviations from Plan
None functionally. The route reservation API internally tracks whether the current call inserted the route so persistence-failure rollback cannot remove a pre-existing idempotent reservation from a concurrent request (an implementation detail within the plan's intent).
Testing
mise run pre-commitpasses (format, license headers, Helm lint) — with one pre-existing, unrelated failure:rust:lockfiles:checkrejectsexamples/supervisor-middleware-content-guard/Cargo.lockbecauseclap 4.6.7was yanked from crates.io; that lockfile is untouched by this PR and fails on unmodifiedmainas well.e2e/were modified.Tests added:
openshell-coreconfig: default/accept/reject validation,with_server_sanspreserves the routing name.openshell-serverservice_routing: fixed hash vectors (enh47uqyj7bpyttz,eiud2z5apv3yeucg), hashed URL construction, index idempotency/removal/collision/rebuild-from-store, configured-hash host resolution for Host header and HTTP/2 authority, legacy-host acceptance when configured, unknown-hash fallthrough.openshell-servergrpc::service:expose_service_rejects_route_hash_collision_before_persistence.openshell-serverconfig_file/cli: TOML parse/serialize, file-populated value, CLI/env override precedence, effective-config reach.openshell-cli:service_url_for_gatewaypreserves the hashed label while normalizing the port.openshell-serverservice_routing tests.Verification runs (local):
cargo fmt --check,git diff --check— passcargo clippy -p openshell-core -p openshell-server -p openshell-cli --all-targets -- -D warnings— passcargo test -p openshell-core— 558 unit + 1 doctest passcargo test -p openshell-server— 2205 pass (20 Postgres-dependent ignored as expected)cargo test -p openshell-cli service_url— passmise run helm:test— 345 chart tests passChecklist
Documentation updated:
docs/how-it-works/gateways/configuration.mdx: new TOML field reference (default, validation, precedence, hash URL example, distinction fromgateway_jwt.gateway_id)docs/how-it-works/gateways/overview.mdx: corrected hostname grammar (workspace segment) + legacy vs hashed formatsdocs/how-it-works/sandboxes/overview.mdx: consume printed URLs, not constructed hostnamesdocs/kubernetes/ingress.mdx: exact-host ingress guidance for multi-gateway shared-wildcard-cert deploymentsdocs/kubernetes/setup.mdx: chart-values table entrydeploy/helm/openshell/README.md: regenerated chart values docs