Skip to content

fix(isthmus): route native-image metadata queries through reflective provider - #1046

Open
nielspardon wants to merge 1 commit into
substrait-io:mainfrom
nielspardon:fix/native-image-metadata-janino
Open

fix(isthmus): route native-image metadata queries through reflective provider#1046
nielspardon wants to merge 1 commit into
substrait-io:mainfrom
nielspardon:fix/native-image-metadata-janino

Conversation

@nielspardon

@nielspardon nielspardon commented Jul 29, 2026

Copy link
Copy Markdown
Member

Problem

Queries like the following crash in the isthmus native image (#251):

./isthmus -c 'CREATE TABLE foo(a VARCHAR);' "SELECT a FROM foo GROUP BY a"
./isthmus -c 'CREATE TABLE foo(a VARCHAR);' "SELECT DISTINCT a FROM foo"

A grouping-only GROUP BY (and SELECT DISTINCT) goes through
RelBuilder.aggregate_alreadyUnique()RelMetadataQuery.areColumnsUnique().
Calcite's default RelMetadataQuery obtains its handlers from
JaninoRelMetadataProvider, which compiles a handler class at runtime with Janino.
Runtime code generation is unavailable in a GraalVM native image (closed-world,
ahead-of-time compiled), so the query fails:

java.lang.AssertionError: Required type "Ljava/lang/AssertionError;" not found
    at org.codehaus.janino.IClassLoader.requireType
    at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile

An aggregate with a function (count(...)) skips the uniqueness check, so the
existing smoke test never exercised the broken path — which is why this stayed
uncaught.

Fix

Calcite ships ProxyingMetadataHandlerProvider, a reflection-based (dynamic-proxy)
alternative to the Janino code generator. Dynamic proxies are supported in native
images once their interfaces are registered. SqlConverterBase already used this
provider — but only the SQL-expression path used that cluster. The SQL-query path
(SubstraitSqlToCalcite) and the Substrait→Calcite path
(ConverterProvider.getRelBuilder, SubstraitRelNodeConverter.convert) built
clusters with no metadata supplier and fell back to Janino.

  • Extract the setup into Utils.useReflectiveMetadataProvider(RelOptCluster) and
    apply it to every cluster / RelBuilder used for Calcite conversion (removing the
    duplicated inline lambda in SqlConverterBase).
  • The reflective provider proxies the bare BuiltInMetadata$* interfaces, not
    just the $Handler sub-interfaces already present, so register those in
    reachability-metadata.json (the safe superset — unused proxy registrations are
    harmless).
  • Add grouping-only GROUP BY and SELECT DISTINCT cases to smoke.sh so the
    native image exercises the uniqueness path in CI.

Closes #251

🤖 Generated with AI

…provider

Grouping-only `GROUP BY` and `SELECT DISTINCT` queries crashed in the
isthmus native image. `RelBuilder.aggregate_` runs a column-uniqueness
metadata query whose default `RelMetadataQuery` obtains handlers from
`JaninoRelMetadataProvider`, which compiles a handler class at runtime with
Janino. Runtime code generation is unavailable in a GraalVM native image
(closed-world, ahead-of-time compiled), so the query failed with a Janino
type-resolution error. An aggregate *with* a function skips the uniqueness
check, so the existing smoke test (which only used `count(...)`) never
exercised the broken path.

`SqlConverterBase` already set a codegen-free `ProxyingMetadataHandlerProvider`
on its cluster, but only the SQL-expression path used that cluster. The
SQL-query path (`SubstraitSqlToCalcite`) and the Substrait->Calcite path
(`ConverterProvider.getRelBuilder`, `SubstraitRelNodeConverter.convert`) built
clusters with no metadata supplier, so they fell back to Janino.

- Extract the proxying-provider setup into `Utils.useReflectiveMetadataProvider`
  and apply it to every cluster/`RelBuilder` used for Calcite conversion.
- The reflective provider proxies the bare `BuiltInMetadata$*` interfaces, not
  just the `$Handler` sub-interfaces already registered, so register those too
  in `reachability-metadata.json`.
- Add grouping-only `GROUP BY` and `SELECT DISTINCT` cases to `smoke.sh` so the
  native image exercises the uniqueness path in CI.

Verified by building the native image locally (GraalVM 25) and running
`smoke.sh` / `tpch_smoke.sh`.

Closes substrait-io#251
@nielspardon
nielspardon marked this pull request as ready for review July 29, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[isthmus] CLI failures in native image due to undefined proxy classes

1 participant