Skip to content

fix(registry): reopen when another caller wins creation - #241

Open
KaiqiJinWow wants to merge 1 commit into
lance-format:mainfrom
KaiqiJinWow:fix/registry-concurrent-create-recovery
Open

fix(registry): reopen when another caller wins creation#241
KaiqiJinWow wants to merge 1 commit into
lance-format:mainfrom
KaiqiJinWow:fix/registry-concurrent-create-recovery

Conversation

@KaiqiJinWow

@KaiqiJinWow KaiqiJinWow commented Sep 9, 2026

Copy link
Copy Markdown

Summary

RolloutRegistry::open_or_create can fail during concurrent initialization. Two callers may both observe DatasetNotFound; if one creates the registry before the other attempts creation, the second caller receives DatasetAlreadyExists and fails startup despite a registry being available. This is what we observed in nightly build.

For example, two server replicas can start against the same new object-store location:

  1. Replica A loads the registry and receives DatasetNotFound.
  2. Replica B loads the registry and also receives DatasetNotFound.
  3. Replica A creates the registry successfully.
  4. Replica B attempts the same create and receives DatasetAlreadyExists.

In this PR, we handle this outcome by reopening the existing registry with the same storage options. Other create and load errors continue to propagate.

Testing

Added a deterministic regression test that simulates this interleaving: both callers observe an absent registry, one creates it and inserts a row, and the other attempts creation through the recovery path. The test checks that the reopened dataset has the expected version and retains the row.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant