Skip to content

fix(bufstream): Clean up etcd PVCs during W&B deletion - #418

Open
danielpanzella wants to merge 4 commits into
mainfrom
danielpanzella/etcd-pvc-purge
Open

danielpanzella wants to merge 4 commits into
mainfrom
danielpanzella/etcd-pvc-purge

Conversation

@danielpanzella

@danielpanzella danielpanzella commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Deleting a W&B CR with retentionPolicy.onDelete: purge can leave Bufstream's etcd PVCs behind because the old claim template lacks the W&B labels used by the purge selector. Separately, the normal read/reconcile path attempts to purge etcd data whenever the broker Application is missing, even during a partial deployment.

This change makes the existing W&B deletion finalizer the only Bufstream PVC cleanup path:

  • Remove PVC deletion from ReadState; a missing broker Application reports a missing resource and preserves etcd data.
  • Add W&B purge labels to new etcd claim templates, and recognize legacy claims using the exact workload labels and generated claim name while rejecting conflicting ownership labels.
  • Preserve detach and per-component retention overrides. A failed PVC deletion keeps the W&B cleanup finalizer so reconciliation can retry.

Legacy claim matching retains the workload-label checks: Kubernetes copies StatefulSet selector labels onto generated PVCs even when the claim template has no labels (upstream controller). This matches the saved WESTest claims; unlabeled, unrelated claims are not deleted by name alone.

Validation:

  • Added a regression test that reproduced deletion of new and legacy claims during normal reconciliation before the fix and now passes.
  • Added tests through the W&B Reconcile deletion path for purge with Applications present/absent, detach, retention overrides, finalizer ordering, and retry after a deletion error.
  • Bufstream and reconciler package suites pass, including the new regressions. go vet ./... passes.
  • make test ran the full suite; its only failure was the unrelated TestTelemetryChartFullModeRendersCoreStack, whose Helm output is missing name: victoria-log-agent.
  • make lint could not complete: installed golangci-lint 2.13.2 cannot decode the Go 1.27.2 export data format (version 5 is greater than maximum supported version 4).

A published operator build and fresh EKS end-to-end validation are still pending. The companion WESTest PR #56 adds PVC/PV draining and run-scoped EBS cleanup.

@danielpanzella
danielpanzella marked this pull request as ready for review October 19, 2026 19:30
@danielpanzella
danielpanzella requested a review from a team as a code owner October 19, 2026 19:30
@danielpanzella danielpanzella changed the title fix(bufstream): purge etcd PVCs with legacy labels fix(bufstream): Purge etcd PVCs with legacy labels Oct 19, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

@danielpanzella danielpanzella changed the title fix(bufstream): Purge etcd PVCs with legacy labels fix(bufstream): Clean up etcd PVCs during W&B deletion Oct 19, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant