Skip to content

fix(db): exclude kind:30179 ciphertext from brownfield FTS - #6822

Open
tlongwell-block wants to merge 1 commit into
mainfrom
meli/fts-exclude-30179
Open

fix(db): exclude kind:30179 ciphertext from brownfield FTS#6822
tlongwell-block wants to merge 1 commit into
mainfrom
meli/fts-exclude-30179

Conversation

@tlongwell-block

Copy link
Copy Markdown
Collaborator

Problem

Kind 30179 (NIP-PMA private managed agent — NIP-44 ciphertext carrying the agent nsec, env vars, prompt) joined AUTHOR_ONLY_KINDS in #4593, but only fresh installs stopped indexing it:

  • migrations/0008 installs the positive FTS allowlist only when events is empty;
  • migrations/0014 wraps the retained brownfield expression for 30350 alone;
  • schema/schema.sql still carries the legacy negative skip-set without 30179.

So a relay upgraded in place keeps tokenizing 30179 ciphertext into events.search_tsv. Found by Wren while reviewing #4999 (which activates publication of 30179 from Desktop). No readable leak: /query applies search_hit_accepted + event_visible_to_reader before serialization and live foreign searches against builderlab returned [] — this is the storage-layer privacy invariant (docs/nips/NIP-PMA.md deployment step 2) plus wasted FTS work.

Fix

  • migrations/0033_private_managed_agent_fts.sql — same shape as 0014: capture the current generated expression via pg_get_expr, drop/re-add search_tsv wrapped with kind = 30179 → NULL. Every other kind keeps whatever policy the database already had (fresh allowlist or brownfield skip-set). Rebuilds the GIN index; no heap-wide policy rewrite.
  • schema/schema.sql — add 30179 to the desired-state skip-set.

Tests

  • populated_upgrade_preserves_search_policy_except_for_private_kinds (renamed from …_push_leases): provisions the legacy negative expression (migrations 1–7), inserts kind 1 / 30179 / 30350 rows, checkpoints after 0014 (30179 still searchable, 30350 not), then runs to head and asserts 30179 is NULL. Proven red with the 0033 file removed: left: [(1, Some(true)), (30179, Some(true)), (30350, None)].
  • embedded_migrator_contains_consolidated_initial_schema: count 32→33, asserts 0033 shape and schema.sql parity.
  • buzz-search/tests/fts_integration.rs setup applies 0033 so the FTS tripwires run against the real chain.

Verification (HEAD 6154cc3, same shell)

  • cargo test -p buzz-db -- --include-ignored --test-threads=1: 305 passed / 3 failed — the 3 (create_community_with_owner_enforces_per_owner_limit, insert_mentions_indexes_rosters_past_bind_parameter_cap, transfer_ownership_returns_limit_reached_for_maxed_transferee) fail identically on pristine main f249710 with a fresh DB; unrelated to this diff.
  • cargo test -p buzz-search -- --include-ignored: 19 passed (incl. author_only_kinds_are_storage_level_unsearchable, p_gated_persistent_kinds_have_storage_null_tsvector).
  • desired_state_schema_bootstrap_progresses_beyond_fencing + run_migrations_applies_consolidated_initial_schema_on_fresh_database: pass.
  • cargo fmt --all --check clean; cargo clippy -p buzz-db -p buzz-search --all-targets -D warnings clean.

Independent of #4999 — both main and #4999 are brownfield-exposed today; this lands either order.

Kind 30179 (NIP-PMA private managed agent) joined AUTHOR_ONLY_KINDS in
#4593, but only fresh installs stopped indexing it: migration 0008's
positive allowlist never touches populated databases, and 0014 wrapped
the retained brownfield expression for 30350 alone. A relay upgraded in
place still tokenizes the NIP-44 ciphertext into events.search_tsv.

Add migration 0033 in the 0014 shape: capture the current generated
expression and re-add search_tsv wrapped with kind = 30179 -> NULL, so
every other kind keeps whatever policy the database already had. Mirror
the exclusion in schema/schema.sql.

Tests: extend the populated-upgrade migration test with a 30179 row and
a pre-0033 checkpoint (fails without the new file); assert 0033's shape
in the embedded-migrator test; apply 0033 in the buzz-search FTS setup.

Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
@tlongwell-block
tlongwell-block requested a review from a team as a code owner August 25, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant