feat(format): add cell preconditions to the transaction format

[lichuang]

feat(format): add cell preconditions to the transaction format

Part of #8976. Vote-only PR: proto change, format specification, and compile-required edits. The builder and commit-loop implementation is intentionally excluded and lands in the follow-up #9118, per the persisted-format change process.

Contract being accepted

A transaction may carry preconditions: declarations that specific cells must be unchanged relative to the transaction's read_version when the commit lands. A cell is a (field, physical row address) pair; each precondition declares a set of cells as the cartesian product of field ids and a serialized row-address set (RowAddrTreeMap).

Motivating case: read-compute-write workloads such as publishing a derived column y = f(x). The staged values are only valid if the cells of x they were computed from still hold the same contents. Conflict compatibility cannot guard this — a read-x-write-y transaction is compatible with a concurrent update of x, and rebasing over it silently publishes stale values.

Semantics fixed by this spec

The specification (docs/src/format/table/transaction.md, new "Preconditions" section) pins the points that must be decided language-agnostically, before any implementation fixes them by accident:

1. Composition — preconditions is repeated and conjunctive: every entry must hold. A violation rejects the commit; it is never rebased over the changed cells.
2. Absent / empty / invalid — absent or empty means no conditions. Entries with empty field_ids or empty rows are degenerate (declare no cells); producers must not emit them. Consumers must reject undecodable rows or negative field ids as corrupt transactions.
3. "Unchanged" — defined conservatively in manifest-structure terms, evaluated against the pinned read_version state: fragment removal, data-file or deletion-file replacement, or any overlay affecting declared cells added / modified / removed / narrowed counts as a change; an overlay already present at the read version and unchanged does not. Implementations may over-reject but must never under-reject.

The rows wire format is fully specified inline (u32 entry count; per fragment: u32 fragment id, u32 bitmap byte length, roaring bitmap of offsets; zero length = whole fragment selected). Stable-row-id preconditions are explicitly reserved for a future revision.

Compatibility

• Proto: additive field (repeated Precondition preconditions = 5 on Transaction; field 5 previously unused). Old readers ignore it; new readers see it absent for old writers. No field numbers reused or retyped.
• The feature is inert until a producer emits it: no behavior change for any existing transaction.
• transaction.proto is a stable persisted contract (transaction logs and inline manifest transactions), so this change requires the format-spec vote.

[github-actions]

Important

Format specification vote

This PR modifies the Lance format specification, so it requires 3 binding +1 votes from PMC members (excluding the proposer) and a minimum 72-hour voting period, weekends excluded, before it can merge. Vote by approving this PR (+1) or requesting changes (−1, a veto). See the voting process.

Status: ❌ Blocked — 0 of 3 required approvals

Approvals (this commit)	none (0/3)
Vetoes	none
Voting period	ends Tue 2026-09-15 05:43 UTC (Mon 22:43 PDT)

_{Updated automatically by the format-spec vote gate, which re-checks every 15 minutes — just voted? Re-check now (press Run workflow; leave the input blank to re-check every open format PR). A PMC member may apply the format-waived label to waive the vote for a trivial edit (typo, wording, formatting).}

[lance-gatekeeper]

✅ Gate recommendation: approve.

The revision closes both contract gaps: row selections are now structured in protobuf with portable Roaring payloads, and the contract requires protected-field existence, type, and nullability to remain unchanged relative to read_version. The additive format remains inert for existing transactions, with implementation correctly sequenced in #9118.

[lichuang]

https://github.com/lance-format/lance/actions/runs/34446896028/job/102773541990?pr=9121 the failed case dataset::scanner::test::test_scan_limit_offset is not related with this pr.

[wkalt]

Is it necessary to consider deletion to be a change? If we declare some cells and unrelated cells in the fragment get deleted then ideally that would not be an issue. Even if the declared cells are deleted, ideally that shouldn't cause recomputation of non-deleted values. The deleted/needlessly-computed values will be masked by the deletion vector.