refactor: drive JSON/CBOR serde off the data with a load-time wire-name map - #351
Merged
Merged
Conversation
jterapin
changed the title
jterapin
force-pushed
the
serde-data-driven
branch
from
August 19, 2026 22:52
4c433ea to
a1dc496
Compare
jterapin
changed the title
jterapin
marked this pull request as ready for review
August 20, 2026 15:55
richardwang1124
approved these changes
Aug 24, 2026
richardwang1124
left a comment
richardwang1124
left a comment
Contributor
There was a problem hiding this comment.
Looks good to me! Do you have any benchmarks you can include to show the performance impact?
Contributor
Author
Good call. Let me try updating an overall performance improvements on the prs where I did the schema extensions improvements. |
jterapin
added a commit
that referenced
this pull request
Aug 25, 2026
…353) ## Context This is the second PR in the stack and should be reviewed on top of #351. PR #351 moved JSON and CBOR structure serde onto the actual input or payload data, but the follow-up lookup path was still protocol-blind and relied on temporary normalization. This PR moves that lookup behavior behind explicit schema and JSON extension helpers. ## What changed - Added `Smithy::Schema::Extension` for generic modeled-member lookup and shared trait helpers. - Added `Smithy::Json::Extension` for `jsonName`-aware lookup used only when JSON serde opts into `json_name` behavior. - Cached generic member lookup indexes on shapes as `shape[:member_index]`. - Cached JSON member lookup indexes on shapes as `shape[:json_index]`. - Cached resolved JSON wire names on members as `member[:json_name]`. - Restored `MemberShape#name` as the modeled member name and updated generated schema emission accordingly. - Kept raw Smithy trait data string-keyed rather than symbol-normalizing it so future dynamic-client paths can continue resolving traits directly from model-shaped data. - Routed document serde `jsonName` handling through the same extension helpers. - Moved generic `sparse?` handling to `Smithy::Schema::Extension`, with JSON delegating to it. ## Behavior after this change - JSON with `json_name: true` uses `jsonName`. - JSON with `json_name: false` uses the modeled member name. - CBOR uses the modeled member name. - Union `__type` is only preserved when it is explicitly modeled through `jsonName`. ## Explicit non-goals - No XML behavior changes in this PR. - No Query behavior changes in this PR. ## Compatibility note `MemberShape#initialize` still accepts `location_name:` as a temporary fallback for downstream stacked work and existing generated projections. ## Validation - `bundle exec rspec gems/smithy-schema/spec gems/smithy-json/spec gems/smithy/spec/interfaces/schema/serde_traits_spec.rb` - `bundle exec smithy build --debug` ## Performance Summary ### JSON Wins And Regressions #### Top JSON Wins - Large AWS JSON `GetItem` response with binary-heavy payload: `16.4%` lower mean latency on Graviton, `10.4%` lower on Intel - Baseline AWS JSON `PutItem` request: `14.0%` lower mean latency on Intel, `13.4%` lower on Graviton - Small AWS JSON `PutItem` request with binary payload: `13.4%` lower mean latency on Intel, `10.2%` lower on Graviton - Baseline AWS JSON `GetItem` response: `12.7%` lower mean latency on Intel, `7.2%` lower on Graviton - Large AWS JSON `GetItem` response: `11.3%` lower mean latency on Graviton, `8.2%` lower on Intel Notes: - The clearest wins are concentrated in AWS JSON `GetItem` / `PutItem` scenarios, especially baseline and binary-heavy payloads. - Intel shows the broader set of `>=5%` wins, while Graviton still improves on the largest JSON cases. - Overall, JSON is the strongest-performing protocol family in this benchmark set. #### Top JSON Regressions - AWS JSON healthcheck example response: `15.4%` higher mean latency on Intel, `10.4%` higher on Graviton - AWS JSON healthcheck example request: `7.4%` higher mean latency on Graviton - Small AWS JSON `GetItem` response payload: `4.6%` higher mean latency on Intel Notes: - The healthcheck scenarios are the only clear JSON regressions above the `+5%` threshold. - Outside those outliers, the remaining JSON regressions are low single-digit and read as near-baseline rather than meaningful movement. ### CBOR Wins And Regressions #### Top CBOR Wins - Small CBOR `PutItem` request with binary payload: `9.2%` lower mean latency on Intel - Baseline CBOR `PutItem` request: `8.4%` lower mean latency on Intel, `6.7%` lower on Graviton - Baseline CBOR `GetItem` response: `6.0%` lower mean latency on Intel, `5.5%` lower on Graviton - Medium nested CBOR `PutItem` request: `5.4%` lower mean latency on Intel Notes: - CBOR shows a handful of real wins, mostly on baseline and `PutItem` request shapes, but the overall effect is modest rather than dramatic. - Across all CBOR serde cases, mean latency is `1.4%` lower and p99 latency is `2.2%` lower, so most results still cluster close to baseline. #### Top CBOR Regressions - No CBOR scenario crosses a `+5%` regression threshold. -- Written with AI assistance and reviewed by jterapin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
This is the first PR in the current schema-serde stack.
V4 currently walks every declared structure member on each request and response and resolves wire names inline. This PR changes JSON and CBOR structure serde to work from the actual payload or input data instead, using a load-time member lookup to recover the same basic shape V3 had.
Follow-up PR #353 builds on this by replacing the temporary global lookup with protocol-aware schema extensions. I still want early feedback on the traversal change itself, which is why I am splitting the work this way.
What changed
StructureShapebuilds a load-time@members_by_wire_nameindex atadd_member, keyed by each member'slocation_nameandjsonNamewhen present.Testing
bundle exec rake smithy-schema:specbundle exec rake smithy-json:specbundle exec rake smithy-cbor:specbundle exec rake smithy:spec:unitbundle exec rake smithy:spec:protocols smithy:rbs:protocolsbundle exec rake smithy-schema:rbsaws-sdk-ruby-stagingprotocol validation green forawsJson1_0,awsJson1_1,restJson1,awsQuery,ec2Query, andrestXmlNotes
jsonNameis only used when the active protocol actually requires it.--
Written with AI assistance and reviewed by jterapin.