HIVE-29672: Extract column stats, constraints, token, and WM logic from ObjectStore#6552
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the standalone metastore RawStore surface by introducing dedicated sub-store interfaces/implementations (e.g., column stats, constraints, WLM) and wiring many former RawStore methods through new default delegations. It also updates metastore tests and verification helpers to exercise the new store boundaries (including DirectSQL vs JDO comparison paths).
Changes:
- Add new metastore sub-store interfaces (
ColStatsStore,ConstraintStore,WLMStore) and implementations (ColStatsStoreImpl,ConstraintStoreImpl,WLMStoreImpl). - Convert a number of
RawStoremethods intodefaultmethods delegating tounwrap(...)stores; adjustCachedStoreand tests accordingly. - Update
VerifyingObjectStoreto verify column-stats and partition-stats behavior viaColStatsStore/TableStoreunder DirectSQL configurator toggling.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java | Switches verification logic to ColStatsStore/TableStore with DirectSQL toggling. |
| standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java | Updates a column-validation test to call ColStatsStoreImpl.validateTableCols. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java | Adds default delegations to ColStatsStore/ConstraintStore/WLMStore via unwrap. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/impl/WLMStoreImpl.java | New WLM store implementation extracted into its own class. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/impl/TableStoreImpl.java | Refactors table/partition conversion helpers and adds getMTable(TableName, ...). |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/impl/ConstraintStoreImpl.java | New constraints store implementation (PK/FK/UK/NN/DC/CC operations). |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/impl/ColStatsStoreImpl.java | New column-stats store implementation (read/update/delete/aggregate paths). |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/iface/WLMStore.java | New WLM store interface with @MetaDescriptor. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/iface/TableStore.java | Adds AttachedMTableInfo and getMTable(TableName, ...). |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/iface/ConstraintStore.java | New constraint store interface with @MetaDescriptor. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/iface/ColStatsStore.java | New column-stats store interface with @MetaDescriptor. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metastore/GetHelper.java | Exposes savepoint helpers (now public) for use by new store implementations. |
| standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java | Removes methods now satisfied by RawStore defaults (delegation via unwrap). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+347
to
+351
| final String fkColType = getColumnFromTableColumns(childCols, fkColumnName).getType(); | ||
| fkSignature.append( | ||
| generateColNameTypeSignature(fkColumnName, fkColType)); | ||
| referencedKSignature.append( | ||
| generateColNameTypeSignature(pkColumnName, fkColType)); |
asf-ci-hive
added
tests pending
tests passed
and removed
tests unstable
tests pending
labels
Member
Author
|
I assume this is the last biggest refactors, @ngsg @deniskuzZ @wecharyu @okumin could you take a look if have secs? Thanks in advance! |
dengzhhu653
changed the title
ngsg
reviewed
Jul 6, 2026
ngsg
left a comment
ngsg
left a comment
Contributor
There was a problem hiding this comment.
I read through the changes. Most of them look like extensions of the previous refactoring patch and seem fine to me. I left comments where I have questions. Could you please take a look? Thank you.
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
|
Member
Author
|
The partition_type_check.q seems flaky, which I couldn't repeat the same locally, filed HIVE-29700 to track it |
Member
Author
|
The recent 3 runs are failed by the same TestMiniLlapLocalCliDriver[partition_type_check], will track this failure in HIVE-29700. Thank you for the review @ngsg @saihemanth-cloudera! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?