fix(db): exclude kind:30179 ciphertext from brownfield FTS - #6822
Open
tlongwell-block wants to merge 1 commit into
Open
fix(db): exclude kind:30179 ciphertext from brownfield FTS#6822tlongwell-block wants to merge 1 commit into
tlongwell-block wants to merge 1 commit into
Conversation
Kind 30179 (NIP-PMA private managed agent) joined AUTHOR_ONLY_KINDS in #4593, but only fresh installs stopped indexing it: migration 0008's positive allowlist never touches populated databases, and 0014 wrapped the retained brownfield expression for 30350 alone. A relay upgraded in place still tokenizes the NIP-44 ciphertext into events.search_tsv. Add migration 0033 in the 0014 shape: capture the current generated expression and re-add search_tsv wrapped with kind = 30179 -> NULL, so every other kind keeps whatever policy the database already had. Mirror the exclusion in schema/schema.sql. Tests: extend the populated-upgrade migration test with a 30179 row and a pre-0033 checkpoint (fails without the new file); assert 0033's shape in the embedded-migrator test; apply 0033 in the buzz-search FTS setup. Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz> Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Kind
30179(NIP-PMA private managed agent — NIP-44 ciphertext carrying the agent nsec, env vars, prompt) joinedAUTHOR_ONLY_KINDSin #4593, but only fresh installs stopped indexing it:migrations/0008installs the positive FTS allowlist only wheneventsis empty;migrations/0014wraps the retained brownfield expression for30350alone;schema/schema.sqlstill carries the legacy negative skip-set without30179.So a relay upgraded in place keeps tokenizing
30179ciphertext intoevents.search_tsv. Found by Wren while reviewing #4999 (which activates publication of 30179 from Desktop). No readable leak:/queryappliessearch_hit_accepted+event_visible_to_readerbefore serialization and live foreign searches against builderlab returned[]— this is the storage-layer privacy invariant (docs/nips/NIP-PMA.mddeployment step 2) plus wasted FTS work.Fix
migrations/0033_private_managed_agent_fts.sql— same shape as 0014: capture the current generated expression viapg_get_expr, drop/re-addsearch_tsvwrapped withkind = 30179 → NULL. Every other kind keeps whatever policy the database already had (fresh allowlist or brownfield skip-set). Rebuilds the GIN index; no heap-wide policy rewrite.schema/schema.sql— add30179to the desired-state skip-set.Tests
populated_upgrade_preserves_search_policy_except_for_private_kinds(renamed from…_push_leases): provisions the legacy negative expression (migrations 1–7), inserts kind 1 / 30179 / 30350 rows, checkpoints after 0014 (30179 still searchable, 30350 not), then runs to head and asserts 30179 is NULL. Proven red with the 0033 file removed:left: [(1, Some(true)), (30179, Some(true)), (30350, None)].embedded_migrator_contains_consolidated_initial_schema: count 32→33, asserts 0033 shape andschema.sqlparity.buzz-search/tests/fts_integration.rssetup applies 0033 so the FTS tripwires run against the real chain.Verification (HEAD 6154cc3, same shell)
cargo test -p buzz-db -- --include-ignored --test-threads=1: 305 passed / 3 failed — the 3 (create_community_with_owner_enforces_per_owner_limit,insert_mentions_indexes_rosters_past_bind_parameter_cap,transfer_ownership_returns_limit_reached_for_maxed_transferee) fail identically on pristinemainf249710 with a fresh DB; unrelated to this diff.cargo test -p buzz-search -- --include-ignored: 19 passed (incl.author_only_kinds_are_storage_level_unsearchable,p_gated_persistent_kinds_have_storage_null_tsvector).desired_state_schema_bootstrap_progresses_beyond_fencing+run_migrations_applies_consolidated_initial_schema_on_fresh_database: pass.cargo fmt --all --checkclean;cargo clippy -p buzz-db -p buzz-search --all-targets -D warningsclean.Independent of #4999 — both main and #4999 are brownfield-exposed today; this lands either order.